mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 08:09:22 +00:00
4ce6a16dbdde888d5fcbd109260a86c8a08e4e3d
2297
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Hengfei YangandAshish Kolhe
|
42e965899b |
feat: support use cookie to do cli command (#8189)
### **User description**
We can use cookie to make requests for CLI command for the case enabled
SSO can't use username & password.
```
ZO_CLI_USER_COOKIE=""
```
For now only used for CLI command.
___
### **PR Type**
Enhancement
___
### **Description**
- Add cookie-based CLI authentication
- Fallback to basic auth when no cookie
- Introduce config `ZO_ROOT_USER_COOKIE`
___
### Diagram Walkthrough
```mermaid
flowchart LR
CFG["Config: add root_user_cookie"] -- "read env ZO_ROOT_USER_COOKIE" --> CLI["CLI HTTP request builder"]
CLI -- "cookie present" --> HDR["Set Cookie header"]
CLI -- "cookie empty" --> BASIC["Use basic_auth(email,password)"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>http.rs</strong><dd><code>Add Cookie header auth with
fallback</code>
</dd></summary>
<hr>
src/cli/basic/http.rs
<ul><li>Read <code>root_user_cookie</code> from config.<br> <li> Use
Cookie header when provided.<br> <li> Fallback to basic auth if cookie
empty.<br> <li> Preserve JSON body handling.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8189/files#diff-790b7780a51e8eecd4754b623af8d15ad87baaaf29455ba646e2e410f83f5498">+7/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>config.rs</strong><dd><code>Support cookie env in auth
config</code>
</dd></summary>
<hr>
src/config/src/config.rs
<ul><li>Add <code>root_user_cookie</code> to <code>Auth</code>
struct.<br> <li> Map to env var <code>ZO_ROOT_USER_COOKIE</code>.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8189/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+2/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Co-authored-by: Ashish Kolhe <ashish.j.kolhe@gmail.com>
|
||
 Hengfei YangandBhargav
|
2c0395124f |
feat: deprecated syslog (#8187)
### **User description** impl #8186 ___ ### **PR Type** Enhancement, Tests ___ ### **Description** - Mark syslog modules as deprecated - Add allow(deprecated) to callers - Block enabling syslog setting - Remove syslog-related tests and OpenAPI types ___ ### Diagram Walkthrough ```mermaid flowchart LR meta["meta::syslog module"] -- "mark deprecated" --> callers["call sites"] callers -- "add allow(deprecated)" --> compile_ok["compiles with warnings suppressed"] db_set["db::syslog::toggle_syslog_setting"] -- "reject enable=true" --> error["returns deprecation error"] openapi["OpenAPI schema"] -- "remove syslog types" --> spec["updated API spec"] tests["syslog tests"] -- "removed" --> leaner["reduced test surface"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>8 files</summary><table> <tr> <td><strong>config.rs</strong><dd><code>Suppress deprecation warnings for syslog globals</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Allow deprecated syslog handlers; annotate endpoints</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-58010c6e78f69dc57c353890a807371cd43c2b4cceeeb1e9378130f26d898397">+6/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Suppress deprecation in API scope setup</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Allow deprecated syslog server wiring</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-66207bd699343342402f88cdd331f4c549783754e4921d0c8d43e026e2d8e575">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Allow deprecated syslog cache/watch tasks</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+4/-0</a> </td> </tr> <tr> <td><strong>syslog_server.rs</strong><dd><code>Allow deprecated syslog server run linkage</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-cb7610142582b44885f673583ab16262c3b9ca6de1f59a9def65e503360665a0">+2/-0</a> </td> </tr> <tr> <td><strong>syslog.rs</strong><dd><code>Allow deprecated syslog ingest; remove tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-4bc46f36fad79f5df8c42b8a8d5d8c4a70b1320b966bcff3421deeb96a1981fa">+3/-14</a> </td> </tr> <tr> <td><strong>syslogs_route.rs</strong><dd><code>Allow deprecated route endpoints; remove tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-325df35568c993fc0727bfb0499b682e6bd30af781687706306b3bdb429c49e2">+7/-52</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>6 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Mark syslog meta module as deprecated</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-fbf797574839e69491ea0429141c038c0ff96297bdf3d8c60d60cdc4acec74b1">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Mark syslog HTTP request module deprecated</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-a1cfd40da1ab6cd4db0181fd8664965034298047b3b7c3320c5385897a7c4aed">+1/-0</a> </td> </tr> <tr> <td><strong>openapi.rs</strong><dd><code>Remove syslog types from OpenAPI imports</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-edb8ce232c0cd08366879f642788a75d99f6bab64b4c0e77391281fdb9241e90">+0/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Deprecate syslog DB module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Deprecate syslog logs module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-ce72aea3fa4c44904cbd9ec07bd844bec71b49c9676d667b14056aa5798f96b2">+1/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Deprecate syslogs_route service module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-1b15316505c31acccb26ba66d920fdb0894567d08d2bf1e3457716d128a67b83">+1/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>syslog.rs</strong><dd><code>Remove unit tests for syslog meta types</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-dc6813e31e32c23aae5760091460d2ec4023d37ae73e49d29052d9f4e9f2e1fa">+0/-127</a> </td> </tr> </table></details></td></tr><tr><td><strong>Error handling</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>syslog.rs</strong><dd><code>Block enabling syslog; allow deprecated routes/cache</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8187/files#diff-4c0e9282e3d2352f9a382099901fe034bae2961eed01d764f16ad52f27f55db4">+11/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: Bhargav <BJP232004@GMAIL.COM> |
||
 Ashish Kolhe
|
2514b88d87 |
fix: skip alert evaluation when stream not found (#8179)
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- Add schema cache lookup helper
- Skip alert evaluation if stream missing
- Improve error message for missing stream
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["Alert evaluation"] -- "resolve stream names" --> B["Per-stream checks"]
B -- "schema missing" --> C["Return error: skip evaluation"]
B -- "schema exists" --> D["Fetch settings & validate period"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Introduce schema cache lookup
utility</code>
</dd></summary>
<hr>
src/infra/src/schema/mod.rs
<ul><li>Add <code>get_stream_schema_from_cache</code> async helper.<br>
<li> Build cache key via <code>mk_key</code> prefix strip.<br> <li>
Return cloned <code>Schema</code> from
<code>STREAM_SCHEMAS_LATEST</code>.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8179/files#diff-386fd4fad1e48ab0a90516cab1947b1888b270b2fe912f30d2255cc38e6c519b">+14/-0</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Bug fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Skip alerts when referenced
stream missing</code>
</dd></summary>
<hr>
src/service/alerts/mod.rs
<ul><li>Validate each referenced stream exists in schema cache.<br> <li>
Short-circuit with clear error to skip evaluation.<br> <li> Keep
existing max query range validation.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8179/files#diff-ba9b65130898444f7815fd6068bdb0dc590612f74cc893f79d91b0afc44cc851">+9/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
 sktrpathi
|
75fc81cecf |
chore: remove bogus unit tests (#8172)
### **User description** A significant chunk of API unit tests were not testing any actual business logic, but performing trivial actions such as struct-field-access, derive trait validation such as `Eq`, `PartialEq`, `Clone`, `Serialize` etc, and many of the tests were also just comparing constant values. They added no value to the test suite and have now been removed from the codebase, making the tests more focused. Additionally, - A lot of flaky tests have been corrected. These used to fail due to occasional race conditions when ran locally. - Some mock setup and teardown functions have also been cleaned up, which makes tests a lot more consistent. ___ ### **PR Type** Tests, Enhancement ___ ### **Description** - Remove trivial, non-value unit tests - Relax flaky assertions and panics handling - Improve test isolation via DB/cache resets - Minor test output formatting improvements ___ ### Diagram Walkthrough ```mermaid flowchart LR tests["Remove trivial/boilerplate tests"] -- cleaner suite --> stability["Stabilize assertions and panics"] stability -- reduce flakiness --> isolation["DB/Cache resets in tests"] isolation -- consistent runs --> outputs["Improve test log formatting"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><details><summary>11 files</summary><table> <tr> <td><strong>organization.rs</strong><dd><code>Remove struct-field and default-value trivial tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-4f089715a7ac4f78b45039d773f2fccbb4fbc6a409e0ab9ceee1fcf7911bc44b">+4/-325</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Drop JSON roundtrip serialization tests; tidy imports</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-c4bd542e4e07ed7706b0643086f5a7d4edbb3647c7ceedbe4bea516c5274d8bd">+1/-183</a> </td> </tr> <tr> <td><strong>auth.rs</strong><dd><code>Remove Debug/PartialEq trivial tests; format strings</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+2/-104</a> </td> </tr> <tr> <td><strong>main.rs</strong><dd><code>Make tracing test tolerant; ignore enterprise tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-42cb6807ad74b3e201c5a7ca98b911c5fa08380e942be6e4ac5807f8377f87fc">+23/-60</a> </td> </tr> <tr> <td><strong>event.rs</strong><dd><code>Remove constants and simple struct creation tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-689769712cbd6bd02beba078c1e1dfc188e2be39ab19a0e232b8bd12bc7f1e78">+1/-95</a> </td> </tr> <tr> <td><strong>validator.rs</strong><dd><code>Ensure test isolation with ORM init and cache clears</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f4d4b51574b378a8c372345ea48875834a4eadacfac33fa4d0ceca1e0ef96b35">+28/-22</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Make base URI tests robust to env errors</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-d7987e62dcc7b1ec1bd4aeabef94f85d8c41b2ea33de5ab78660794fa4d51a9e">+25/-4</a> </td> </tr> <tr> <td><strong>users.rs</strong><dd><code>Use best-effort cleanup; clear caches; improve logs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-98da2588a28d62bb2e301d989836125f89f89b76272cd7f75e08ec4deeea5f38">+12/-9</a> </td> </tr> <tr> <td><strong>integration_test.rs</strong><dd><code>Compact println formatting and panic messages</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-7ff815b68943f08bef79f083c09e55ca8c18db8e25033034b12fa31f1d43dab4">+3/-3</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Remove strict zero-length assertions on globals</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-cda0d9179ed982fa5f0989c9b26f21cfa49dd1de28195a98135650ffcc7c602a">+1/-13</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Remove unused constant import in tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-a5ea87a7766cb24619f64a6100f385f4b42c47af7d33b8e2f060229c536d7418">+1/-8</a> </td> </tr> </table></details></td></tr><tr><td><strong>Additional files</strong></td><td><details><summary>26 files</summary><table> <tr> <td><strong>authz.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-28a815d1c4728aff854a9996de0ba3d3dafe1bea6bfb0cf2a9f4186c40f985b6">+0/-52</a> </td> </tr> <tr> <td><strong>loki.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-121c166af5eab8b9cb83716afa7b65fa7856c006c809b3d910ced184c52ec587">+0/-7</a> </td> </tr> <tr> <td><strong>proxy.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-7f12a4f2483e737671ac41f9c35b411e7323e0ff2731ad92a3b309e144d1316a">+0/-46</a> </td> </tr> <tr> <td><strong>service.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-61d456e7217d19bcec3920a4d1c17fd396a257c81e5394655db3632a057bdd9e">+0/-7</a> </td> </tr> <tr> <td><strong>service_account.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-878ad520e1c84500cc21ff6fc4088c8da64ff2febd79e60c0d4cc8c695c49798">+0/-80</a> </td> </tr> <tr> <td><strong>traces.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-bda8182c1f17a8360337c769edade0cb9478ff1443dd6aedce7f526afbe97afe">+0/-106</a> </td> </tr> <tr> <td><strong>user.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-e13b3a981fd70182b9738a6b0ecfdee915c5b36757703a87741e1eddf3f6e786">+0/-101</a> </td> </tr> <tr> <td><strong>auth_tests.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2688cbdc0b009b3f0dd829c69ab86c6a7d0778ab3e65e916a410a76aa6ae6480">+0/-2961</a></td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2aa59ce4464e7dbabdb9367fc8bb8c1b489c914b8690577d4916870463ab12db">+0/-1</a> </td> </tr> <tr> <td><strong>action.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-fc87c75a7df9539d7e7ee6184dfab02d4a8932323e81f95d57c05363aa44752d">+0/-18</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-5bf716db7e1d91dfbd427a2e320ce407c4f0c92d55d68f8e636daed1b02abd56">+0/-13</a> </td> </tr> <tr> <td><strong>reports.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-d2117a3d67d4ec47b0017861845c92143baf917e10e6746df5dd97f925f81eeb">+0/-10</a> </td> </tr> <tr> <td><strong>destinations.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-e0ba33d95ef7f0eac13bdac7e750462907a0308f01c027ea206d058e282c977b">+0/-103</a> </td> </tr> <tr> <td><strong>inverted_index.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-8bf9d9403f05f53f0c6fd5e6a6dbf0204ea80465b233567552cb7395d08d1c3c">+0/-22</a> </td> </tr> <tr> <td><strong>meta_store.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-45018c838da377fa9ded9305a1ee752adde06acdb607addf26c95f5747537615">+0/-6</a> </td> </tr> <tr> <td><strong>ratelimit.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-fa7dc03fb526c09d5eb58021872d821fcd72eeb4d6a670224f5286012485bc3d">+0/-10</a> </td> </tr> <tr> <td><strong>timed_annotations.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f94b4d6d12d68e7054ad33601e666b08e4594586013dc34384fa21b1df5fe6e8">+0/-12</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+0/-34</a> </td> </tr> <tr> <td><strong>m20250611_000002_populate_reports_table.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-8247af6d6f8ccb6a5a51daff4cb1e352585be2d1c785d86ae5ebfea1dbc1099c">+0/-1</a> </td> </tr> <tr> <td><strong>folders.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-402482c22fda5dff8f33382ac11d0908f693975dec7c08b37b1c5fe24a20486a">+0/-1</a> </td> </tr> <tr> <td><strong>bulk.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2017dbe9076c78a7e7919ed5c8738729123d630038803cef0ce2c356d6fa2c48">+0/-9</a> </td> </tr> <tr> <td><strong>ingestion.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-3edd5f4ed1eed0f765b7b450ef0fbf587632e6b22302006a5e397049e297057f">+0/-7</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-2f7f0bac01afc6e37908521d1d33d05fcd650b7a08792062976e1e7465b4a1f9">+0/-42</a> </td> </tr> <tr> <td><strong>footer_cache.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-f9278e53ed7131c953fc30f8b15cda801b74b81fb2d570fe7f80a3b7fc1de872">+0/-7</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-1602f85399080c1e2838d1ef1d62af23d96f353379e49a7770026e892c954018">+0/-20</a> </td> </tr> <tr> <td><strong>mod.rs</strong></td> <td><a href="https://github.com/openobserve/openobserve/pull/8172/files#diff-66f3b4f5e7927092fd6e57738687a1091b6f316638f7c58f8d7be9addb72bc30">+0/-31</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
 Loakesh Indiran
|
087b921dd4 |
fix: unify cache delay secs across result cache, aggs cache and metrics cache (#8171)
## New Env: - `ZO_CACHE_DELAY_SECS` default 300 secs (5mins) ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Rename retention config to cache-centric - Unify delay logic across subsystems - Update env var to ZO_CACHE_DELAY_SECS - Align WAL, compaction, caching timings ___ ### Diagram Walkthrough ```mermaid flowchart LR Config["Config Limit: cache_delay_secs"] --> WAL["WAL retention and flush timing"] Config --> Ingester["Ingester flush loop"] Config --> ParquetMove["Parquet move age checks"] Config --> Compaction["Merge/downsample scheduling windows"] Config --> PromQLCache["PromQL cache/store time windows"] Config --> GRPCSearch["gRPC search need_wal decision"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>7 files</summary><table> <tr> <td><strong>wal.rs</strong><dd><code>Replace retention time with cache delay in WAL expiry</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8171/files#diff-4928ae81597b28facdc3644f786ac14c60b0a406e1c7b8eac7232fbafbcbb6e3">+2/-2</a> </td> </tr> <tr> <td><strong>lib.rs</strong><dd><code>Use cache_delay_secs for periodic flush interval</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8171/files#diff-603267e2f712eeb4bc46fdff981f1fad1f3a9d3b989a7db4b3c0984fc5b1ed43">+1/-1</a> </td> </tr> <tr> <td><strong>writer.rs</strong><dd><code>Apply cache_delay_secs to WAL age threshold</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8171/files#diff-d1a9b1542cc2758afe4fa67abd454dd818718f4e1c7570303519af5a58d6301f">+1/-1</a> </td> </tr> <tr> <td><strong>parquet.rs</strong><dd><code>Use cache_delay_secs in file age selection</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8171/files#diff-27f710981cfe1b938aeeb53abaed05961596c32e2edf3b8a2a59d3e72b2f542b">+1/-1</a> </td> </tr> <tr> <td><strong>merge.rs</strong><dd><code>Unify compaction scheduling windows with cache_delay_secs</code></dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8171/files#diff-0ab016dbd5129c9c285c9c459bf78928147bce97dc92a03aa754da3da71168d7">+2/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Use cache_delay_secs for need_wal decision</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8171/files#diff-53b070c152a50ae30476e20bb02779a6b2b84b604f8610cb6604307acd931668">+1/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Unify cluster need_wal cutoff using cache_delay_secs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8171/files#diff-63910f18561d2d84326bc989187e779e2bf1ecd02b2310da55632210810f4d89">+1/-3</a> </td> </tr> </table></details></td></tr><tr><td><strong>Configuration changes</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>config.rs</strong><dd><code>Rename config field and env var to cache_delay_secs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8171/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+2/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Align cache set window and tests to cache_delay_secs</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8171/files#diff-697dba801a55938e37c08346a3fbfad808edf0eb47b7cb4d65b4d2ac6843577e">+2/-2</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
Ashish Kolhe
|
7bda973b58 |
fix: sql generation for quick alerts (#8169)
### **PR Type**
Bug fix
___
### **Description**
- Fix WHERE placement in grouped alert SQL
- Ensure filter applied before GROUP BY/HAVING
- Minor spacing cleanup in non-grouped query
___
### Diagram Walkthrough
```mermaid
flowchart LR
buildSQL["build_sql in alerts/mod.rs"]
groupCase["Grouped query path"]
noGroupCase["Non-grouped query path"]
bug["WHERE placed after FROM"]
fix["Move WHERE before GROUP BY/HAVING"]
buildSQL -- "when group_by present" --> groupCase
groupCase -- "was" --> bug
bug -- "now" --> fix
buildSQL -- "when no group_by" --> noGroupCase
noGroupCase -- "retain WHERE placement" --> fix
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Bug
fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Correct SQL clause ordering
for grouped alert queries</code>
</dd></summary>
<hr>
src/service/alerts/mod.rs
<ul><li>In grouped path, insert WHERE before GROUP BY/HAVING.<br> <li>
Replace extraneous space and ensure proper clause order.<br> <li> Keep
WHERE inclusion in non-grouped path; minor spacing tweak.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8169/files#diff-ba9b65130898444f7815fd6068bdb0dc590612f74cc893f79d91b0afc44cc851">+2/-2</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
 Huaijin
|
dfa041446e | fix: cache with distinct and explain panic (#8166) | ||
 udev
|
b4eac8f607 |
fix: Inc UT coverage (#8128)
increase UT coverage improve the email regex to catch `doubledotsequence` in email |
||
|
Hengfei Yang
|
48115e1ab9 |
fix: enhance machine id check (#8160)
### **User description** - Add duplicated machine id check - Add 4 random chars for generate parquet filename - Use Atomic replace unsafe code ___ ### **PR Type** Bug fix, Enhancement ___ ### **Description** - Centralize node ID generation logic - Replace unsafe globals with atomics - Detect and resolve duplicate node IDs - Truncate long Snowflake IDs when parsing ___ ### Diagram Walkthrough ```mermaid flowchart LR A["Cluster register (etcd/nats)"] -- "collect node ids" --> B["generate_node_id()"] B -- "new id" --> C["LOCAL_NODE_ID (AtomicI32)"] C -- "reload if changed" --> D["Snowflake ID generator"] E["set_status()"] -- "detect duplicate id" --> B F["CLI parse-id"] -- "truncate to 19 digits" --> G["parse i64"] H["Parquet filename"] -- "append 4-hex random" --> I["unique file path"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>cli.rs</strong><dd><code>Truncate IDs longer than 19 digits before parse</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+8/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>10 files</summary><table> <tr> <td><strong>etcd.rs</strong><dd><code>Use centralized node-id generation and atomics</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-37c2c2803d1c12ebb4ef9299bc79c9e6740fc5f19f7f4d84e7cfbb610b9561a9">+3/-17</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add reusable generate_node_id utility</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+17/-0</a> </td> </tr> <tr> <td><strong>nats.rs</strong><dd><code>Centralize id generation; detect duplicates; reload</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-901773980aa255942e7865b977fe63811f31511ac40bc512d6bbb14dbb54bece">+20/-17</a> </td> </tr> <tr> <td><strong>cluster.rs</strong><dd><code>Replace unsafe globals with AtomicI32</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-68d7195ab7cccf22579c4944a37c8a2f592392c30371ce907fadf62fca6f97dd">+3/-3</a> </td> </tr> <tr> <td><strong>ider.rs</strong><dd><code>Read machine id atomically; support reload</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-6ea42b8eab8ba0708f93f9a8b44056d9a9e6ac42517ffba12f81420d6b0bd2b0">+10/-2</a> </td> </tr> <tr> <td><strong>cluster.rs</strong><dd><code>Node equality includes id; NodeStatus i32</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-a2667c1be522ce095796a7066e02631a7e01acc85a46573e1e3a157d2aefb7c3">+4/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add 4-hex random suffix to filenames</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-30f42c3685037c219f0033617f032b23a7f74624ba4ae6c4e3687af401304591">+2/-1</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Initialize generators using atomic node id</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-4696a44abde2df4ac48fed716f930bfdcc01759bdfcfd8d802821c6dd857bff9">+7/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Generate record ids using atomic node id</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+2/-2</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Use atomic node id for stream generators</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8160/files#diff-d9596182a0c7a46a6a7ff5cee822554cdae05ceec312f86c93b538db9f73bbfd">+7/-2</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
Loakesh Indiran
|
c78f58325a |
fix: ingester writer channel closed (#8155)
### **User description**
- During ingestion there are instances where the writer channel is
closed. And this will need an app restart.
- So instead when we do get_writer check if channel is closed and if it
is creates a new writer
___
### **PR Type**
Bug fix
___
### **Description**
- Detect closed writer channels and refresh
- Remove stale writers from cache on close
- Add channel-closed helper on Writer
- Improve rotate error log with writer index
___
### Diagram Walkthrough
```mermaid
flowchart LR
GW["get_writer()"] -- "cache hit + channel open" --> RET["return cached Writer"]
GW -- "cache hit but channel closed" --> RM["remove from cache"]
RM -- "proceed to slow path" --> CREATE["create new Writer"]
GW -- "cache miss" --> CREATE
W["Writer"] -- "is_channel_closed()" --> GW
TTL["check_ttl()"] -- "log with idx on error" --> LOG["enhanced error log"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Bug
fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>writer.rs</strong><dd><code>Handle closed writer
channels and improve logging</code>
</dd></summary>
<hr>
src/ingester/src/writer.rs
<ul><li>Check cached writer channel state before returning.<br> <li>
Remove closed-channel writers from cache and recreate.<br> <li> Add
Writer::is_channel_closed helper method.<br> <li> Enhance rotate error
log to include writer index.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8155/files#diff-d1a9b1542cc2758afe4fa67abd454dd818718f4e1c7570303519af5a58d6301f">+22/-2</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
sktrpathi
|
05d0c67704 |
chore: add UTs, run fmt and clippy (#8153)
### **User description** - also increase minimum coverage threshold ___ ### **PR Type** Tests, Enhancement ___ ### **Description** - Add extensive unit tests for metrics - Add comprehensive tests for logs bulk API - Add tests for traces response and helpers - Increase minimum coverage thresholds |
||
 harish0101-ops
|
d1657b077d | fix: 8107 Correct alert URL query for condition (#8143) | ||
|
Huaijin
|
d0fab62ee4 |
chore: add metrics for custom ExecutionPlan (#8131)
|
||
|
udev
|
77e50caea5 |
fix: make config reactive to env changes (#7996) (#8092)
### **User description** fix #7996 sync to main ___ ### **PR Type** Bug fix, Enhancement ___ ### **Description** - Make periodic jobs react to config changes - Introduce reusable `spawn_pausable_job!` macro - Fix env var name `ZO_SEARCH_JOB_RUN_TIMEOUT` - Replace fixed intervals with dynamic getters ___ ### Diagram Walkthrough ```mermaid flowchart LR CFG["get_config() (dynamic)"] MACRO["spawn_pausable_job! macro"] JOBS["Background jobs (GC, compactor, alerts, stats, etc.)"] ENV["Env vars (e.g., ZO_SEARCH_JOB_RUN_TIMEOUT)"] ENV -- "drives" --> CFG CFG -- "intervals/pause_if" --> MACRO MACRO -- "spawns/controls" --> JOBS ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><details><summary>2 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Heartbeat interval now read each loop</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-d15434a27e2b8406811f5a707b605203b603dad4a262c0642326be860bd2b568">+2/-1</a> </td> </tr> <tr> <td><strong>config.rs</strong><dd><code>Fix search job timeout env var name</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Enhancement</strong></td><td><details><summary>16 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Export pausable_job utilities module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-f5df40ca59915d9d58ad04ea2440e6ebf29388afaeb8cb0d6c63d2a979df0c17">+1/-0</a> </td> </tr> <tr> <td><strong>pausable_job.rs</strong><dd><code>Add pausable job spawning macro and helpers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-7e2dce7a3942789804bae659939c214c4eeb68d89315e23a08139214c339ea24">+143/-0</a> </td> </tr> <tr> <td><strong>disk.rs</strong><dd><code>Use pausable job for disk cache GC</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-af63f2c5d85313779df0204c9689e50a6c3916e3aadc24e54e23ec640a584fcb">+4/-10</a> </td> </tr> <tr> <td><strong>memory.rs</strong><dd><code>Use pausable job for memory cache GC</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-ff00d697a9b223ef2858c5e41111a8918e7cc347dc22891c02bda6982682eee1">+4/-16</a> </td> </tr> <tr> <td><strong>lib.rs</strong><dd><code>Mem persist uses dynamic sleep interval</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-603267e2f712eeb4bc46fdff981f1fad1f3a9d3b989a7db4b3c0984fc5b1ed43">+4/-5</a> </td> </tr> <tr> <td><strong>alert_manager.rs</strong><dd><code>Make alert/search jobs pausable and reactive</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-7e30858abad652a860186eb047347305488f52356112b1495dd53ce740316d7c">+61/-104</a></td> </tr> <tr> <td><strong>compactor.rs</strong><dd><code>Refactor compactor jobs to pausable macro</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-550629b7ebf6460b36fdd46cdfea466c3d3f1e30023c71ee54b87e7998efc500">+134/-188</a></td> </tr> <tr> <td><strong>parquet.rs</strong><dd><code>Use dynamic interval for file push loop</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-27f710981cfe1b938aeeb53abaed05961596c32e2edf3b8a2a59d3e72b2f542b">+1/-1</a> </td> </tr> <tr> <td><strong>flatten_compactor.rs</strong><dd><code>Flatten generator via pausable macro</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-dd30b917380ba55bd17aadeaee6aa2c87a67e861257a6a14ba2113a9b3015030">+15/-20</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire jobs with reactive spawning; telemetry</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+21/-13</a> </td> </tr> <tr> <td><strong>promql.rs</strong><dd><code>Make metrics leader push pausable/optional</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-15414a41d64eb87bc5c0cd00a07213b3f99c31b346e8e2c36178f7276a1f058c">+26/-31</a> </td> </tr> <tr> <td><strong>stats.rs</strong><dd><code>Stats jobs via pausable macro; conditional start</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-f3ca67d9d2c68beeafd6e5de947dbc9a648ac2d2699c5dc87458c369887d09b9">+51/-42</a> </td> </tr> <tr> <td><strong>telemetry.rs</strong><dd><code>Remove legacy telemetry loop (replaced)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-e82cf6d4c890bfaa5e711293b599d7f7a742bb96180639815ede33958b1bc1d9">+0/-36</a> </td> </tr> <tr> <td><strong>storage.rs</strong><dd><code>Make enrichment merge job pausable</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-335bb1eea9193dc398da6838c6a6a11dd6f09e984a1ed4a2e611618a427754f0">+42/-35</a> </td> </tr> <tr> <td><strong>distinct_values.rs</strong><dd><code>Periodic flush via pausable macro</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-3fb2320197c6e8e900658f5a24158e922610eadcbf081ab0df230f770ef27e07">+11/-18</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Audit publish via pausable macro, optional run</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8092/files#diff-c6255010f7a87d6acdb3ad04954fc9f2c27350ae319c53a1857b120aa06408a3">+17/-15</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
Ashish Kolhe
|
ebe5c0e73e |
feat: search time patterns (#8104)
### **PR Type** Enhancement, Tests, Documentation ___ ### **Description** - Apply regex patterns at search time - Map projections to source columns - Integrate enterprise pattern manager - Add tests for projection extraction ___ ### Diagram Walkthrough ```mermaid flowchart LR SearchReq["Search request"] -- "execute query" --> DatafusionPlan["DataFusion logical plan"] DatafusionPlan -- "extract projection mappings" --> ProjMap["ProjectionColumnMapping"] SearchReq -- "fetch hits" --> Hits["Search results (hits)"] ProjMap -- "guide field processing" --> PatternMgr["Pattern manager (enterprise)"] Hits -- "apply regex patterns" --> Hits2["Transformed results"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>6 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Expose projections module in meta</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8104/files#diff-bab79e8da2d0d9f903cb1a6187d922c6de63d3e4561e8ce633660b664b4e859f">+1/-0</a> </td> </tr> <tr> <td><strong>projections.rs</strong><dd><code>Add ProjectionColumnMapping structure</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8104/files#diff-2432b13a92a92cdbd1aa1ee0fbdde05827a67c37081e7f0ecc127a4effbe3101">+26/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Post-process hits with regex patterns</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8104/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+61/-2</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Register regex_projections planner module</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8104/files#diff-70bb941c2eef1365e5ead1a59b65c5397457b9197a4e22596e18d17db7eaf922">+1/-0</a> </td> </tr> <tr> <td><strong>cache.rs</strong><dd><code>Apply regex patterns to cached responses</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8104/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+10/-0</a> </td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Apply regex patterns in streaming paths</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8104/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+35/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>regex_projections.rs</strong><dd><code>Extract projection source columns with tests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8104/files#diff-b329e80dfa025fabb0ef51b6dc25285aaadc00ab7862157deb0eb7b0b01bb494">+500/-0</a> </td> </tr> </table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>AssociatedRegexPatterns.vue</strong><dd><code>Enable UI section for detect-at choice</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8104/files#diff-c6fce5e1c83798991b6ff19effb23c67af59f38fac0b9e56dcab7a09d139f550">+1/-1</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
Huaijin
|
cde4ba7e83 | fix: tantivy result cache to access plan (#8126) | ||
|
sktrpathi
|
459d901e9b |
chore: add UTs, run fmt and clipyy (#8122)
### **PR Type** Tests, Enhancement |
||
|
sktrpathi
|
9a11a680ae |
chore: add test cases (#8095)
### **User description** - also clear over 400 clippy issues ___ ### **PR Type** Tests, Enhancement ___ ### **Description** Add extensive unit tests for enrichment storage Expose helpers for testability in storage module Modernize tests: clippy-friendly string formatting Normalize constants in time-based tests |
||
|
Huaijin
|
20a5fce67c |
feat: improve tracing for flight (#8097)
- [x] add span for tantivy_search - [x] improve tracing for flight after this pr the tracing for arrow flight will like below <img width="1770" height="506" alt="image" src="https://github.com/user-attachments/assets/55e30700-122c-4dec-bff7-8eed0f823bce" /> |
||
|
Huaijin
|
f73c29f333 | chore: add unit test (#8099) | ||
 ktx-vaidehiandLoakesh Indiran
|
9a422cb34d |
feat: dashboard unique colors to the table column automatically. (#8051)
### **PR Type**
Enhancement
___
### **Description**
- Add unique value auto-color for tables
- Introduce typed override config schema
- Update popup UI for new config types
- Pass color mode through table pipeline
___
### Diagram Walkthrough
```mermaid
flowchart LR
Schema["Dashboard v5 Config enum"]
Popup["OverrideConfigPopup UI logic"]
Convert["convertTableData column metadata"]
Renderer["TableRenderer cell style"]
Palette["colorPalette getColorForTable"]
Schema -- "new type 'unique_value_color'" --> Popup
Popup -- "persist {type, autoColor|value}" --> Convert
Convert -- "set col.colorMode='auto'" --> Renderer
Renderer -- "map unique values to colors" --> Palette
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Typed dashboard Config with
color variant</code>
</dd></summary>
<hr>
src/config/src/meta/dashboards/v5/mod.rs
<ul><li>Replace <code>Config</code> struct with tagged enum.<br> <li>
Add variants: <code>Unit{value?}</code> and
<code>UniqueValueColor{auto_color}</code>.<br> <li> Adjust serde to use
<code>type</code> tag and camelCase.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8051/files#diff-da49585bbcb02bf0f737bb68739ebc248edd15c7ad9427c0e68f7728bb4a8c17">+13/-8</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>OverrideConfigPopup.vue</strong><dd><code>Popup
supports unit and auto color overrides</code>
</dd></summary>
<hr>
web/src/components/dashboards/OverrideConfigPopup.vue
<ul><li>Redesign popup to choose config type.<br> <li> Support unit
config and unique value color (auto).<br> <li> Normalize, transform, and
save typed configs.<br> <li> UI tweaks: width, layout, and
handlers.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8051/files#diff-f1512a15069c23bcc71ac88db5b0ab882031bf6ec9d53a84de2fcd8adb6dec2c">+155/-62</a></td>
</tr>
<tr>
<td>
<details>
<summary><strong>OverrideConfig.vue</strong><dd><code>Adapt prop shape
for typed overrides</code>
</dd></summary>
<hr>
web/src/components/dashboards/addPanel/OverrideConfig.vue
<ul><li>Pass override configs via <code>{ overrideConfigs }</code>
shape.<br> <li> Simplify dialog prop wiring.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8051/files#diff-54a6038f5c684ade27b88e3c6a134874b51501904441fa7849a532d10aa7d533">+3/-5</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>TableRenderer.vue</strong><dd><code>Table cells
auto-colored by unique values</code>
</dd></summary>
<hr>
web/src/components/dashboards/panels/TableRenderer.vue
<ul><li>Add auto-color rendering path for columns.<br> <li> Use
<code>getColorForTable</code> palette and per-column cache.<br> <li>
Fallback to value-mapping color.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8051/files#diff-794706ea9da3a70012a0327c0464f9fd16802f78ece26055fe951716c2d73d59">+24/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>colorPalette.ts</strong><dd><code>Add table color
palette export</code>
</dd></summary>
<hr>
web/src/utils/dashboard/colorPalette.ts
<ul><li>Export <code>getColorForTable</code> palette for tables.<br>
<li> Provide light pastel color list.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8051/files#diff-30ac812743f2388c4061647fac5e9f413bfe561f1fc5d4f6f7be82fce73f2840">+27/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>convertTableData.ts</strong><dd><code>Pipe override
color/unit configs to table</code>
</dd></summary>
<hr>
web/src/utils/dashboard/convertTableData.ts
<ul><li>Parse typed override configs to maps.<br> <li> Set
<code>colorMode: 'auto'</code> on columns when enabled.<br> <li> Use
unit config map for formatting.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8051/files#diff-a672094c46e697ee7b7a3949a901cc1f7f88ec06cddaf67370474b508d5b0960">+32/-8</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
---------
Co-authored-by: Loakesh Indiran <66478092+Loaki07@users.noreply.github.com>
|
||
|
Hengfei Yang
|
0c01a83a27 |
fix: data retention with extended retention (#8089)
### **User description**
Fix the case:
It caused add a lot of retention jobs again and again with extended data
retention.
___
### **PR Type**
Bug fix
___
### **Description**
Add optional date_range to get_min_date
Restrict min date query within range
Avoid duplicate retention jobs generation
Improve delete_by_date logging clarity
___
### Diagram Walkthrough
```mermaid
flowchart LR
FLT["FileList trait get_min_date(date_range)"] -- "implemented in" --> MySQL["MySQL get_min_date with optional range"]
FLT -- "implemented in" --> Postgres["Postgres get_min_date with optional range"]
FLT -- "implemented in" --> SQLite["SQLite get_min_date with optional range"]
Ret["Retention generator"] -- "queries min date None/ranged" --> FLT
Ret -- "skips empty ranges" --> Jobs["No duplicate retention jobs"]
Del["delete_by_date"] -- "logs readable date range" --> Logs["Improved logs"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Add optional date_range to
FileList trait</code>
</dd></summary>
<hr>
src/infra/src/file_list/mod.rs
<ul><li>Extend trait get_min_date with date_range<br> <li> Update public
wrapper to pass date_range</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8089/files#diff-3df833bc826eb12e0819620d1fe316ef1ac352d293c11e57cc3df0da3c031e93">+5/-1</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>mysql.rs</strong><dd><code>MySQL get_min_date supports
optional date range</code>
</dd></summary>
<hr>
src/infra/src/file_list/mysql.rs
<ul><li>Implement ranged MIN(date) query when date_range provided<br>
<li> Fallback to original MIN(date) when None</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8089/files#diff-06d02c738c6a4dc22ff022c16befef8d46aece6cfaa88bb4a5d3d5f92c2e50a1">+17/-5</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>postgres.rs</strong><dd><code>Postgres get_min_date
adds optional range</code>
</dd></summary>
<hr>
src/infra/src/file_list/postgres.rs
<ul><li>Add conditional MIN(date) with BETWEEN-like constraints<br> <li>
Preserve original behavior when no range</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8089/files#diff-bb24a30c9c3744eb31fb49e4271a9651d1f9a94a87455ac0074364b879af9651">+17/-5</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>sqlite.rs</strong><dd><code>SQLite get_min_date
supports optional range</code>
</dd></summary>
<hr>
src/infra/src/file_list/sqlite.rs
<ul><li>Add ranged MIN(date) query variant<br> <li> Keep default
MIN(date) when no range</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8089/files#diff-85ff00530831fdaa4e13c05cb649f2e20479cce04a4b65ef78d131cdb89b6c71">+17/-5</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Bug fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>retention.rs</strong><dd><code>Prevent duplicate
retention jobs via ranged min-date</code>
</dd></summary>
<hr>
src/service/compact/retention.rs
<ul><li>Query min_date with None initially<br> <li> Re-check min_date
within each deletion range<br> <li> Skip ranges without data to prevent
duplicates<br> <li> Improve delete_by_date log with human-readable
range</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8089/files#diff-6abd048f5ef4e7163cffbb51cb4bee10c07fb88b871469d5b3724f7cd04743dc">+28/-7</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Huaijin
|
6042eee547 | chore: add node_name to analyze (#8081) | ||
|
Huaijin
|
0581f626e5 | feat: impl distribute query analyze (#8063) | ||
|
Hengfei Yang
|
c52d6fb473 | fix: improve error msg of nats (#8073) | ||
|
75f2235d04 |
feat: create stream with field and data type (#7851)
### **User description** closes #6660 ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add field data types and UDS support - Validate field names with strict regex - Store defined schema as objects, not strings - Backward-compatible parsing and UI updates ___ ### Diagram Walkthrough ```mermaid flowchart LR API["API: Stream settings/schema"] --> Parse["Parse defined_schema_fields (strings|objects)"] Parse -- "normalize to DataField{name,type}" --> Settings["StreamSettings.defined_schema_fields: DataField[]"] Settings -- "new stream" --> SchemaAdd["Add fields to Arrow schema with types"] Settings -- "existing + UDS" --> UDSUpdate["Add new fields to UDS + persist"] Types["DataType enum (Utf8, Int64, Uint64, Float64, Boolean)"] --> Arrow["Arrow DataType mapping"] Validate["Regex field name validation"] --> Save["save/update settings endpoints"] UI["UI: Add/Edit stream fields"] --> ObjectFmt["Send/handle {name,type} objects"] ObjectFmt -- "compat" --> Compat["Store original + derive string list for display"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>10 files</summary><table> <tr> <td><strong>stream.rs</strong><dd><code>Rename stream property field to raw type key</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-6f5ee19bbcc9cc7fb699fc1c0f43fc96d42698ae1168a2827b5c65f7948d5dbc">+5/-6</a> </td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Introduce DataType and DataField; settings parsing</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-2438d04c3fe97bc5452e5b74a303163b8723d93455cda32c95f0525435ad46bd">+99/-13</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Optimize settings fetch; adapt defined fields helpers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-386fd4fad1e48ab0a90516cab1947b1888b270b2fe912f30d2255cc38e6c519b">+13/-6</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Use DataField names when building UDS sets</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+6/-1</a> </td> </tr> <tr> <td><strong>schema.rs</strong><dd><code>Add DataField flows; new/UDS logic and conversions</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-d9596182a0c7a46a6a7ff5cee822554cdae05ceec312f86c93b538db9f73bbfd">+270/-18</a></td> </tr> <tr> <td><strong>stream.rs</strong><dd><code>Validate field names; resolve defined field types</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-41452a7fb29b4634437174fd6aecf6a9050dbfdef41b9077a21d59af3d79e9ed">+126/-7</a> </td> </tr> <tr> <td><strong>AddStream.vue</strong><dd><code>Send defined fields as objects with type</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-78b8c84a558e60d942bacffee365633d074042e464be465eb6984b0ee5df82b9">+6/-2</a> </td> </tr> <tr> <td><strong>StreamFieldInputs.vue</strong><dd><code>UI for field data type and UX tweaks</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-c8173413939f320f197b0f8790e619da18666f79009345f72b3325e89bbb7566">+90/-27</a> </td> </tr> <tr> <td><strong>schema.vue</strong><dd><code>Support object-based defined fields; sync logic</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-9196be2ccae4054a00b0b1aa9d1e87b366c10920d6eaf9ef8c6e6293d50bcf46">+119/-25</a></td> </tr> <tr> <td><strong>useStreams.ts</strong><dd><code>Compare/sync object-based defined fields</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-a083369344f022cdf8d4f536719179b1d920369e33e2e26234e70dd36d151a7c">+50/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Map UDS fields by object name not string</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7851/files#diff-619eafba0f0509d905b021f4a16659010aca23c62cffd9cd3e6c2bbe204ce200">+1/-1</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai> Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
|
Hengfei Yang
|
b2767cdebc | feat: create data retention by day (#8058) | ||
|
Ashish Kolhe
|
1ddf812acd |
chore: add unit tests (#8061)
### **PR Type**
Tests
___
### **Description**
- Add comprehensive unit tests for quantile
- Expand tests for trend and regression
- Add exhaustive resets function tests
- Add string_to_array_v2 implementation tests
___
### Diagram Walkthrough
```mermaid
flowchart LR
promql_common_tests["Add tests: quantile, trend, regression, kahan"] -- "strengthen coverage" --> promql_common_module["promql/common.rs"]
resets_tests["Add extensive resets tests"] -- "validate edge cases" --> resets_module["promql/functions/resets.rs"]
udf_tests["Add string_to_array_v2 tests"] -- "validate parsing, args" --> udf_module["search/datafusion/udf/string_to_array_v2_udf.rs"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>common.rs</strong><dd><code>Comprehensive statistical
utilities test suite</code>
</dd></summary>
<hr>
src/service/promql/common.rs
<ul><li>Add comprehensive tests for quantile function.<br> <li> Add
tests for calculate_trend logic and edge cases.<br> <li> Add
linear_regression tests including negatives/infinities.<br> <li> Add
kahan_sum_increment precision and usage tests.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8061/files#diff-0661c489ae1b0457aa7b182590eed83315a682f4836fff734cfcb8c1afae86bc">+160/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>resets.rs</strong><dd><code>Extensive resets exec
behavior tests</code>
</dd></summary>
<hr>
src/service/promql/functions/resets.rs
<ul><li>Add unit tests covering no, single, multiple resets.<br> <li>
Test edge cases: empty, single, negatives, NaN/Infinity.<br> <li> Add
helper to build RangeValue for tests.<br> <li> Validate counter overflow
scenarios.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8061/files#diff-96d475ef838ab215c6b259c32d7591ed5e64fb142d79d3f8647adbd57072a3eb">+143/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>string_to_array_v2_udf.rs</strong><dd><code>Broad tests
for string_to_array_v2 implementation</code>
</dd></summary>
<hr>
src/service/search/datafusion/udf/string_to_array_v2_udf.rs
<ul><li>Add tests for basic splitting and batch processing.<br> <li>
Cover edge cases: empty/null delimiter, unicode.<br> <li> Validate
multiple delimiters and empty parts handling.<br> <li> Test wrong arg
count error messaging.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8061/files#diff-1fa623e7d0c4c8485c1a16de78ad4f28e44818b66cf32553525ea189d717ba0d">+182/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
 Subhra264andudev
|
c46fe2bf8c |
fix: new sso admin/viewer/editor should be able to enjoy given permissions (#8002)
Currently, openfga returns error when new sso user (admin/editor/viewer) tries to login into openobserve. This pr fixes that. --------- Co-authored-by: udev <uddhav@openobserve.ai> |
||
|
Hengfei Yang
|
d646bcba8b | fix: add logs for compactor (#8045) | ||
 Shivank Anchal
|
111a67b73d | fix: return proper error for incorrect function panic (#7708) (#8021) | ||
|
ktx-vaidehi
|
cd00d5b06c |
feat: dashboard color based on series name (#7941)
### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add "Color by series" UI and schema - Apply custom series colors in charts - Emit and wire series data for config - Improve autocomplete filtering robustness ___ ### Diagram Walkthrough ```mermaid flowchart LR UI["ConfigPanel adds ColorBySeries UI"] -- "saves mappings" --> Schema["color.colorBySeries in panel schema"] Charts["PanelSchemaRenderer emits series data"] -- "series-data-update" --> Pages["Visualize/Metrics/AddPanel pages"] Pages -- "pass seriesData" --> UI Colors["applySeriesColorMappings + getSeriesColor"] -- "resolve colors" --> ECharts["ECharts series options"] ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>13 files</summary><table> <tr> <td><strong>mod.rs</strong><dd><code>Add colorBySeries schema and struct</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-da49585bbcb02bf0f737bb68739ebc248edd15c7ad9427c0e68f7728bb4a8c17">+14/-0</a> </td> </tr> <tr> <td><strong>PanelSchemaRenderer.vue</strong><dd><code>Emit series data updates to parents</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-e5376e7e77bf3aaf6590c2522d8c2dbffcc95a8006ae0d6dc0040a37825367d6">+6/-0</a> </td> </tr> <tr> <td><strong>ColorBySeries.vue</strong><dd><code>New Color by series config entrypoint</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-2783259507b45f0b3664d9be68ca236690bbbf151472da3fa8b552d230af5b3f">+126/-0</a> </td> </tr> <tr> <td><strong>ColorBySeriesPopUp.vue</strong><dd><code>New popup to map series to colors</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-8b4dce28a203ce7e70550d6437f16e7661d5a4ecaf336de3450a8dc7fbe5e27a">+385/-0</a> </td> </tr> <tr> <td><strong>ConfigPanel.vue</strong><dd><code>Integrate ColorBySeries and accept panelData</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-c3a6e64dff02b58bb7cdc3f0668990b41ff41544b4c5e991bbcd989ad4bba3f8">+5/-3</a> </td> </tr> <tr> <td><strong>VisualizeLogsQuery.vue</strong><dd><code>Wire series-data-update and pass to ConfigPanel</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-979b01d943d58c5d70faeeac5948b7f02f0b2392a12b84ef07ba7f7393ab0261">+34/-7</a> </td> </tr> <tr> <td><strong>Index.vue</strong><dd><code>Capture series data and pass to ConfigPanel</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-a6734928ea4dcb5244e90de5a16fe6f5cafa95ccf58430ac7b548a065daf3d61">+11/-0</a> </td> </tr> <tr> <td><strong>AddPanel.vue</strong><dd><code>Capture panel series data for config</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-c9632db1ad95af95aba10161bcbc7c5608318919f228637bdafae3f3a31267e8">+11/-1</a> </td> </tr> <tr> <td><strong>useDashboardPanel.ts</strong><dd><code>Add defaults for mappings and colorBySeries</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-fbba1700750053b9eaf38ddd30dab84531bc44131c795ebb713811e4930cd7cb">+2/-0</a> </td> </tr> <tr> <td><strong>colorPalette.ts</strong><dd><code>Prefer custom colorBySeries mapping</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-30ac812743f2388c4061647fac5e9f413bfe561f1fc5d4f6f7be82fce73f2840">+11/-0</a> </td> </tr> <tr> <td><strong>convertDataIntoUnitValue.ts</strong><dd><code>Add applySeriesColorMappings helper</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-14c6fe442ef231566ec411f9e0262cdd24ec912fc67b9154eec743282ab7211a">+82/-0</a> </td> </tr> <tr> <td><strong>convertPromQLData.ts</strong><dd><code>Apply mapped colors to PromQL charts</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-68d70ba6c622911d48cdb567940dc404a4a6468b86c6d24fb54e5f1d8aa5ca2a">+9/-0</a> </td> </tr> <tr> <td><strong>convertSQLData.ts</strong><dd><code>Apply mapped colors to SQL charts</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-88c0e865040a20b2c8bb7d77e7f6bd187a5b07c1645c120223bc9d59603752c3">+13/-1</a> </td> </tr> </table></details></td></tr><tr><td><strong>Formatting</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>CommonAutoComplete.vue</strong><dd><code>Fix watch comma and cleanup</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-35f59b61336a5ba3f5933f6803e8e2cc84ced59c3f0bea25f370bb2e3d21849b">+1/-2</a> </td> </tr> </table></details></td></tr><tr><td><strong>Bug fix</strong></td><td><details><summary>1 files</summary><table> <tr> <td><strong>useSearchInputUsingRegex.ts</strong><dd><code>Guard nulls in option filtering</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/7941/files#diff-f901991c2e465430e9b2434405852ac225ff94f5adfcfeb0ad0e80fdaadd0f4b">+6/-2</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ |
||
|
Ashish Kolhe
|
3458678f3a |
feat: Query Optimiser cli (#7745) (#7748)
### **PR Type**
Enhancement
___
### **Description**
- Add `query-optimiser` subcommand with required flags
- Dispatch to `query_optimiser` handler in CLI
- Introduce `query_optimiser` module with enterprise logic
- Fallback stub when enterprise feature disabled
___
### Diagram Walkthrough
```mermaid
flowchart LR
CLIApp["CLI Application"] --> Dispatch["Subcommand Dispatch"]
Dispatch -- "query-optimiser" --> QueryOpt["query_optimiser Module"]
QueryOpt -- "feature enterprise" --> GetRec["get_recommendations API"]
QueryOpt -- "no enterprise" --> Fallback["Unsupported Handler"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>cli.rs</strong><dd><code>Add query-optimiser command
handling</code>
</dd></summary>
<hr>
src/cli/basic/cli.rs
<ul><li>Added <code>query-optimiser</code> command definition<br> <li>
Defined arguments: url, token, duration, etc.<br> <li> Implemented match
branch for dispatch</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7748/files#diff-cb1eb3e74723d891353ccf1e2e5e0d246d2e78a5e69dbe4c73006ea8412d65e0">+70/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>query_optimiser.rs</strong><dd><code>Implement
query_optimiser functionality</code>
</dd></summary>
<hr>
src/cli/basic/query_optimiser.rs
<ul><li>Created <code>query_optimiser</code> module file<br> <li>
Enterprise feature calls <code>get_recommendations</code><br> <li>
Fallback prints unsupported message</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7748/files#diff-69a90f896bb4c373e3f710b55453404c9cfd1d4b17a8d57a835e2b07de9a85fe">+64/-0</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Register query_optimiser
module</code>
</dd></summary>
<hr>
src/cli/basic/mod.rs
- Registered new `query_optimiser` module
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7748/files#diff-8820944bd26c070b4d7d80c21d699937e32f8f8d0f3dad496edf580bd83aa321">+1/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
  Yashodhan JoshiandYashodhan Joshi
|
d28b5064d8 |
fix: handle non-empty base-uri in router (main) (#8032)
### **User description** cherry pick of https://github.com/openobserve/openobserve/pull/8016 on main, with added tests. ___ ### **PR Type** Bug fix, Tests ___ ### **Description** - Respect non-empty base URI in routing - Update dispatcher to pass base URI - Strip base URI before route matching - Add tests covering base URI logic ___ ### Diagram Walkthrough ```mermaid flowchart LR Req["Incoming request path"] -- includes base URI --> Strip["Strip base_uri prefix"] Strip -- api_path --> Match["Match querier/ingester routes"] Match -- node discovery --> Proxy["Proxy to appropriate node"] Tests["Tests"] -- validate --> Strip ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><table> <tr> <td> <details> <summary><strong>mod.rs</strong><dd><code>Base URI-aware routing and tests</code> </dd></summary> <hr> src/router/http/mod.rs <ul><li>Pass <code>cfg.common.base_uri</code> into <code>get_url</code>.<br> <li> Update <code>get_url</code> to strip <code>base_uri</code> from paths.<br> <li> Base path used for <code>is_querier_route</code> now <code>api_path</code>.<br> <li> Add async test <code>test_with_base_uri</code> for base URI cases.</ul> </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/8032/files#diff-d7987e62dcc7b1ec1bd4aeabef94f85d8c41b2ea33de5ab78660794fa4d51a9e">+22/-3</a> </td> </tr> </table></td></tr></tr></tbody></table> </details> ___ --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com> |
||
|
Huaijin
|
a99527ce52 | fix: tantivy row_id to AccessPlan (#8034) | ||
|
Huaijin
|
c51142de03 | fix: streaming aggregate config (#8029) | ||
|
Ashish Kolhe
|
2ce1fba5d3 |
fix: add max bytes for nats stream (#8009)
### **PR Type**
Enhancement, Bug fix
___
### **Description**
- Add NATS queue max size config
- Validate and default NATS size to 2GB
- Convert GB setting to bytes at init
- Apply max_bytes to JetStream stream
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["Config: Add `queue_max_size` (GB)"] -- "init validation" --> B["`check_nats_config`: default 2GB if 0"]
B -- "convert to bytes" --> C["Store bytes in cfg.nats.queue_max_size"]
C -- "use in stream config" --> D["NATS JetStream: set `max_bytes`"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>config.rs</strong><dd><code>Introduce and validate NATS
`queue_max_size`</code>
</dd></summary>
<hr>
src/config/src/config.rs
<ul><li>Add <code>queue_max_size</code> env config (GB, default 0)<br>
<li> Validate NATS config, default to 2GB if 0<br> <li> Convert size
from GB to bytes during init<br> <li> Wire
<code>check_nats_config</code> into global init</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8009/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+19/-0</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>nats.rs</strong><dd><code>Use config to set JetStream
max_bytes</code>
</dd></summary>
<hr>
src/infra/src/queue/nats.rs
<ul><li>Apply <code>cfg.nats.queue_max_size</code> to JetStream
<code>max_bytes</code><br> <li> Enforce retention by size alongside age
limits</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8009/files#diff-b05729b03dff9a5e29eaf536d0f90e94a0c27a27d1a3a41b52f25dc6dc380138">+1/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Huaijin
|
3eb271620a | refactor: try move remotescan, stream aggregate, topk from rewrite to optimize rule (#7998) | ||
   
|
1d0490875b |
fix: is_multi_stream_search and enable_align_histogram (#8003)
### **User description** Fixes: - https://github.com/openobserve/openobserve/issues/7986 ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add multi-stream search histogram support - Gate histogram alignment via query flag - Remove unused search_type plumbing - Propagate multi-stream flag through services ___ ### Diagram Walkthrough ```mermaid flowchart LR HTTP["HTTP handlers"] GRPC["gRPC handlers"] Utils["HTTP query utils"] SQL["SQL utils: histogram eligibility"] Hist["SQL: histogram query builder"] Svc["Search service"] Cache["Search cache"] Stream["Streaming pipeline"] Utils -- "parse is_multi_stream_search / enable_align_histogram" --> HTTP Utils -- "parse flags" --> GRPC HTTP -- "pass flags" --> Svc GRPC -- "pass flags" --> Svc Svc -- "eligibility check (multi/unified)" --> SQL Svc -- "partitioning uses align flag" --> Svc Svc -- "search() with multi-stream flag" --> Cache Cache -- "eligibility(query, multi)" --> SQL HTTP -- "convert_to_histogram_query(..., multi)" --> Hist Stream -- "do_search(..., multi)" --> Cache Hist -- "build single/multi-stream histogram query" --> Svc ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>15 files</summary><table> <tr> <td><strong>http.rs</strong><dd><code>Parse multi-stream and align histogram flags</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+20/-0</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Remove SearchPartitionRequest.search_type field</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+0/-3</a> </td> </tr> <tr> <td><strong>sql.rs</strong><dd><code>Add multi-stream histogram eligibility (UNION ALL)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+134/-38</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Pass multi-stream flag into search service</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire flags; histogram conversion uses multi-stream</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-7</a> </td> </tr> <tr> <td><strong>multi_streams.rs</strong><dd><code>Partition multi: add align flag; remove search_type</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+10/-9</a> </td> </tr> <tr> <td><strong>search_inspector.rs</strong><dd><code>Pass multi-stream flag in search profile</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+1/-0</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Stream API supports multi-stream histogram and flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+9/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Propagate multi-stream flag in traces search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Cache search considers multi-stream eligibility</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+2/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Align partitions via flag; multi partition API updates</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+22/-12</a> </td> </tr> <tr> <td><strong>histogram.rs</strong><dd><code>Build histogram for multi-stream queries (UNION ALL)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+64/-8</a> </td> </tr> <tr> <td><strong>cache.rs</strong><dd><code>Streaming cache handles multi-stream flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+3/-0</a> </td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Execution path passes multi-stream flag to searches</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+25/-4</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Stream request processing threads multi-stream flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+4/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add multi-stream histogram query support - Align histogram via query flag - Remove search_type plumbing in partitions - Propagate multi-stream flag end-to-end ___ ### Diagram Walkthrough ```mermaid flowchart LR Http["HTTP handlers"] Grpc["gRPC handlers"] Utils["HTTP query utils"] Front["Web client"] Service["Search service"] Cache["Search cache"] SQLU["SQL utils: histogram eligibility"] Hist["Histogram query builder"] Front -- "is_multi_stream, align flag" --> Http Utils -- "parse is_multi_stream/align" --> Http Http -- "enable_align_histogram" --> Service Http -- "is_multi_stream_search" --> Cache Grpc -- "flags default false" --> Service Cache -- "eligibility(sql, multi)" --> SQLU Http -- "convert_to_histogram_query(..., multi)" --> Hist Service -- "partition(gen interval if enabled)" --> Service ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>19 files</summary><table> <tr> <td><strong>http.rs</strong><dd><code>Parse multi-stream and align-histogram flags</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+20/-0</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Remove search_type from partition request</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+0/-4</a> </td> </tr> <tr> <td><strong>sql.rs</strong><dd><code>Histogram eligibility with multi-stream UNION ALL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+134/-38</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Pass default multi-stream flag to service</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add flags to HTTP search; align histogram handling</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-7</a> </td> </tr> <tr> <td><strong>multi_streams.rs</strong><dd><code>Partition multi endpoint supports align flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+10/-9</a> </td> </tr> <tr> <td><strong>search_inspector.rs</strong><dd><code>Forward multi-stream flag in inspector search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+1/-0</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Streaming search supports multi-stream histogram</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+9/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Forward multi-stream flag in traces helpers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Eligibility check aware of multi-stream</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+2/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Align histogram in partition; multi aggregate</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+22/-12</a> </td> </tr> <tr> <td><strong>histogram.rs</strong><dd><code>Build single/multi-stream histogram SQL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+64/-8</a> </td> </tr> <tr> <td><strong>cache.rs</strong><dd><code>Propagate multi-stream flag in streaming cache</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+3/-0</a> </td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Pass multi flag through streaming execution</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+25/-4</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire multi-stream flag across streaming path</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+4/-0</a> </td> </tr> <tr> <td><strong>IndexList.vue</strong><dd><code>Use align flag in partition requests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-08f058f883b659e713e606991a7ea37ae1ff016b7cccef970489ab5840dc3d09">+1/-1</a> </td> </tr> <tr> <td><strong>usePanelDataLoader.ts</strong><dd><code>Toggle align histogram for dashboards</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+3/-1</a> </td> </tr> <tr> <td><strong>useLogs.ts</strong><dd><code>Multi-stream histogram flow and params</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+3/-2</a> </td> </tr> <tr> <td><strong>search.ts</strong><dd><code>API params for is_multi_stream and align flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+7/-5</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: Bhargav <BJP232004@GMAIL.COM> Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> Co-authored-by: omkark06 <omkar@zinclabs.io> Co-authored-by: ktx-akshay <akshay.patil@kiara.tech> |
||
|
Subhra264
|
cb07c8af7e |
fix: improve alert manager error handling (#7936)
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- Add robust trigger usage reporting on failures
- Improve retry/backoff and next-run updates
- Respect alert/report realtime and free-trial states
- Add scheduler debug logs and report success log
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["Scheduler workers"] -- process --> B["Alert triggers"]
A -- process --> C["Report triggers"]
B -- failures/conditions --> D["Update trigger (status/next_run/retries)"]
C -- failures/conditions --> D
B -- publish --> E["Trigger usage telemetry"]
C -- publish --> E
A -- keep-alive --> F["Job keep-alive logs"]
G["Report sender"] -- success --> H["Info log: report sent"]
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>handlers.rs</strong><dd><code>Stronger trigger failure
handling, retries, and telemetry</code></dd></summary>
<hr>
src/service/alerts/scheduler/handlers.rs
<ul><li>Compute queue timing and source node metadata<br> <li> Publish
TriggerData on alert/report fetch failures<br> <li> Refactor trigger
updates: retries, next_run_at, realtime<br> <li> Handle free-trial:
reschedule next run appropriately</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7936/files#diff-5eca8995a88fcecb6d6b8efd3c7c4496f4a96a4b060fb4ed85593574f7cf49eb">+151/-29</a></td>
</tr>
<tr>
<td>
<details>
<summary><strong>reports.rs</strong><dd><code>Log report send
success</code>
</dd></summary>
<hr>
src/service/dashboards/reports.rs
- Log info on successful report send
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7936/files#diff-07dd8993c92038fa1197f9a4b6fe141d310854fbc5859d76ee8723fb90871312">+1/-0</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Documentation</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>worker.rs</strong><dd><code>Add detailed scheduler
debug logs</code>
</dd></summary>
<hr>
src/service/alerts/scheduler/worker.rs
<ul><li>Add debug log after trigger processing<br> <li> Add debug log
for keep-alive extension</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7936/files#diff-3d37991258d3f26b16ad99bba59b63adfa614bfbc4866c4088544ca9308ae49d">+10/-0</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Ashish Kolhe
|
db454f2d50 |
fix: use uds fields length for file move (#7813)
### **PR Type**
Bug fix
___
### **Description**
- Pass UDS schema field count to merge_files
- Compute `num_uds_fields` from stream settings
- Use UDS count when determining field number
- Fallback to full schema fields if none defined
___
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Bug
fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>parquet.rs</strong><dd><code>Add and use UDS fields
count in merge logic</code>
</dd></summary>
<hr>
src/job/files/parquet.rs
<ul><li>Extract <code>num_uds_fields</code> from
<code>stream_settings</code><br> <li> Extend <code>merge_files</code>
signature with <code>num_uds_fields</code><br> <li> Use
<code>num_uds_fields</code> over schema fields when > 0<br> <li> Retain
default schema field count otherwise</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7813/files#diff-27f710981cfe1b938aeeb53abaed05961596c32e2edf3b8a2a59d3e72b2f542b">+10/-1</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Ashish Kolhe
|
cd08a60baf |
fix: vrl error report fn name (#7968) (#7973)
### **PR Type**
Bug fix, Enhancement
___
### **Description**
- Include function name in node errors
- Propagate optional fn name through channel
- Truncate VRL error messages to 500 chars
- Minor refactors in VRL apply path
___
### Diagram Walkthrough
```mermaid
flowchart LR
A["process_node errors"] -- "send (node_id, type, msg, fn)" --> B["error channel tuple +fn"]
B -- "collect" --> C["PipelineError.add_node_error(..., fn)"]
C -- "store" --> D["NodeErrors.function_name (Option)"]
A -- "VRL errors truncated" --> B
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>error.rs</strong><dd><code>Add function name to node
error metadata</code>
</dd></summary>
<hr>
src/config/src/meta/self_reporting/error.rs
<ul><li>Add <code>function_name</code> to <code>NodeErrors</code>
(optional).<br> <li> Extend <code>add_node_error</code> to accept
<code>fn_name</code>.<br> <li> Update <code>NodeErrors::new</code> to
take <code>function_name</code>.<br> <li> Fix tests to include
<code>function_name: None</code>.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7973/files#diff-00a00a557343475be7131843f748fbae2d0c1a4041a5c4dddec56c940a03e613">+13/-3</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Bug fix</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>batch_execution.rs</strong><dd><code>Propagate fn name
and trim VRL error messages</code>
</dd></summary>
<hr>
src/service/pipeline/batch_execution.rs
<ul><li>Extend error channel to carry optional function name.<br> <li>
Pass function name for FunctionNode errors.<br> <li> Truncate VRL error
messages to 500 chars.<br> <li> Minor VRL apply and stream_name handling
tweaks.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7973/files#diff-3f6824a6eddaf1603e5cb0f1a4311ac6a45e1ffb967266ccc1f0bba944625e69">+60/-35</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Formatting</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Minor formatting
cleanup</code>
</dd></summary>
<hr>
src/service/self_reporting/mod.rs
- Remove superfluous blank line.
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7973/files#diff-c6255010f7a87d6acdb3ad04954fc9f2c27350ae319c53a1857b120aa06408a3">+0/-1</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
 Yuta YamaguchiandHengfei Yang
|
6921d9ad96 |
refactor: remove warn(dead_code) annotation from Immutable (#8006)
Remove unnecessary `#[warn(dead_code)]` annotation from `Immutable` struct. Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com> |
||
|
Hengfei Yang
|
9f5aa745a7 |
feat: remove tantivy footer cache (#8004)
### **User description**
We can't control the memory with the cache, it doesn't only cache footer
it will cache all the data, it caused memory issue. so i removed this
feature.
___
### **PR Type**
Enhancement, Bug fix
___
### **Description**
- Remove Tantivy reader cache feature
- Drop related config flag and module
- Rename result cache static to `GLOBAL_CACHE`
- Expose result cache size and memory stats
___
### Diagram Walkthrough
```mermaid
flowchart LR
cfg["Config: remove inverted index reader cache flag"]
http["HTTP status: report Tantivy result cache stats"]
grpc_use["gRPC search: remove reader cache usage"]
resultCache["Result cache: rename to GLOBAL_CACHE + stats"]
module["Puffin directory: remove reader_cache module"]
cfg -- "delete ZO_INVERTED_INDEX_CACHE_ENABLED" --> grpc_use
grpc_use -- "open index per request (no reader cache)" --> module
grpc_use -- "use GLOBAL_CACHE get/put" --> resultCache
http -- "display len + memory_size" --> resultCache
module -- "cleanup module export" --> ( )
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>config.rs</strong><dd><code>Drop inverted index reader
cache config flag</code>
</dd></summary>
<hr>
src/config/src/config.rs
<ul><li>Remove <code>inverted_index_cache_enabled</code> env config.<br>
<li> Delete flag <code>ZO_INVERTED_INDEX_CACHE_ENABLED</code>.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+0/-6</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Expose Tantivy result cache
metrics in status</code>
</dd></summary>
<hr>
src/handler/http/request/status/mod.rs
<ul><li>Replace reader cache stats with result cache stats.<br> <li>
Import <code>grpc::tantivy_result_cache</code>.<br> <li> Report
<code>len</code> and <code>memory_size</code> for result cache.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-ca09c9c9b020049bdbc3382c466bfcca80daac92e98760d9a9a76172abf34049">+8/-3</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>storage.rs</strong><dd><code>Remove reader cache; use
GLOBAL_CACHE for results</code>
</dd></summary>
<hr>
src/service/search/grpc/storage.rs
<ul><li>Remove reader cache usage and logic paths.<br> <li> Always open
index and reader directly.<br> <li> Switch to
<code>tantivy_result_cache::GLOBAL_CACHE</code>.<br> <li> Clean up
imports for removed reader_cache.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-ab2d867e1e2e3dd8307505c2e130039628a84f8131a9b38cd2e7b4e1c141e094">+29/-45</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>tantivy_result_cache.rs</strong><dd><code>Result cache
rename and stats helpers</code>
</dd></summary>
<hr>
src/service/search/grpc/tantivy_result_cache.rs
<ul><li>Rename global static to <code>GLOBAL_CACHE</code>.<br> <li> Add
<code>len()</code> and <code>memory_size()</code> helpers.<br> <li>
Update test to use <code>GLOBAL_CACHE</code>.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-f4bcf95cba21da14d907701d1440b1be9c9140c23de8681639770364e78af01e">+13/-2</a>
</td>
</tr>
<tr>
<td>
<details>
<summary><strong>reader_cache.rs</strong><dd><code>Delete Tantivy reader
cache module</code>
</dd></summary>
<hr>
src/service/tantivy/puffin_directory/reader_cache.rs
- Delete reader cache implementation and tests.
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-baabb8633f8966f2407c724632e85a34f469a1591d52ded16a6d66c40950c190">+0/-423</a>
</td>
</tr>
</table></td></tr><tr><td><strong>Miscellaneous</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>mod.rs</strong><dd><code>Remove reader_cache module
reference</code>
</dd></summary>
<hr>
src/service/tantivy/puffin_directory/mod.rs
- Remove `reader_cache` module export.
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/8004/files#diff-1602f85399080c1e2838d1ef1d62af23d96f353379e49a7770026e892c954018">+0/-1</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
|
||
|
Hengfei Yang
|
1f15b7ed58 | ci: bump version (#7992) | ||
  
|
748cca3ecb |
feat: visualization enhancement (#7456)
### **User description**
### Which query will be used?
- If query is empty will use histogram query
- Else query will be available:
- If query has group by clause :- will use same query
- Else will use histogram query
___
### **PR Type**
Enhancement, Tests, Bug fix
___
### **Description**
- Add server API to extract result schema
- Implement DataFusion plan visitor for projections
- Integrate visualization flow with schema and histogram
- Numerous UI fixes and UX improvements
___
### Diagram Walkthrough
```mermaid
flowchart LR
DF["DataFusion Plan Visitor (Rust)"]
API["HTTP API /result_schema (Rust)"]
FE["Logs Visualize UI (Vue)"]
Loader["Panel Data Loader (TS)"]
Cache["Histogram Handler & Cache (Rust)"]
DF -- "extract projections/group_by/timeseries" --> API
API -- "JSON ResultSchemaResponse" --> FE
FE -- "build config, set fields, decide histogram" --> Loader
Loader -- "inject searchResponse / is_ui_histogram" --> FE
Cache -- "normalize histogram()" --> DF
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>14
files</summary><table>
<tr>
<td><strong>projections.rs</strong><dd><code>Add DataFusion logical plan
visitor to extract schema</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-7a0420d3649da44b80be36d7f1043d6ff81d88395c961266882c3c700f7edff7">+651/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Add /result_schema endpoint and
wire parsing/security</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+224/-2</a>
</td>
</tr>
<tr>
<td><strong>cacher.rs</strong><dd><code>Improve histogram() handling and
make public</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-8c387b22b2c0a282600ee41ca702b3db60f5a284d507c0feb7418608ff062c6c">+22/-11</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Export projections
module</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-70bb941c2eef1365e5ead1a59b65c5397457b9197a4e22596e18d17db7eaf922">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>Index.vue</strong><dd><code>Integrate visualization with
schema API and histogram</code> </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-0323c3da69b369843c5d72bf4df5b00ca6b5147192a213baf331659ecf92af62">+564/-154</a></td>
</tr>
<tr>
<td><strong>VisualizeLogsQuery.vue</strong><dd><code>Field list UX,
props for searchResponse/histogram</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-979b01d943d58c5d70faeeac5948b7f02f0b2392a12b84ef07ba7f7393ab0261">+236/-174</a></td>
</tr>
<tr>
<td><strong>SearchBar.vue</strong><dd><code>Visualize toggle gating,
SQL/VRL disabled states</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-81893de4edf186b0556f610acb280c87baa2ffe67d3b48a60a866094c7635baf">+102/-52</a></td>
</tr>
<tr>
<td><strong>FieldList.vue</strong><dd><code>Readonly/visibility options
for logs visualize</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-a071c06e3e26629cb4ae4751450acc39c61fa49b07a1cc4be3711697829668d4">+13/-1</a>
</td>
</tr>
<tr>
<td><strong>TransformSelector.vue</strong><dd><code>Disable transform
controls in visualize mode</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-0037d8dd9cfbacc26309b26e98ae953e7ccbd0891ac2fd01c6314ebd7eed9d2c">+11/-4</a>
</td>
</tr>
<tr>
<td><strong>ChartSelection.vue</strong><dd><code>Allow restricting
selectable chart types</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-384ce178717731e2c85f1312f12a54b42ceabd0b86f7e2befb0e421ef568a6a3">+14/-11</a>
</td>
</tr>
<tr>
<td><strong>PanelSchemaRenderer.vue</strong><dd><code>Accept
searchResponse and is_ui_histogram props</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-e5376e7e77bf3aaf6590c2522d8c2dbffcc95a8006ae0d6dc0040a37825367d6">+13/-0</a>
</td>
</tr>
<tr>
<td><strong>usePanelDataLoader.ts</strong><dd><code>Support injected
searchResponse and ui histogram</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+30/-5</a>
</td>
</tr>
<tr>
<td><strong>visualizationUtils.ts</strong><dd><code>Add SQL alias
validator for visualization</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-c2c391afcaa0b44c33d3f4fa531a4f31f044ae036f0baa182c8c88e83de066c5">+80/-0</a>
</td>
</tr>
<tr>
<td><strong>query.ts</strong><dd><code>Extend response metadata for
visualization</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-12dd5b3bb9741bbeaa348c0efbbd745d0a582899678738159c6927d34a35191d">+2/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>mod.rs</strong><dd><code>Add test for compact delete value
check</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-b7e3bf7f0ac23dd084c0c817a310a4d5e7a64ff532746d182986ef940fa03fd0">+17/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>2 files</summary><table>
<tr>
<td><strong>auth.rs</strong><dd><code>Allow auth bypass for
result_schema route</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-9bc2b1841504a435f66ad094f72ddbe7b7fa75e5e1557b8adbfe36f7f0592e1b">+3/-0</a>
</td>
</tr>
<tr>
<td><strong>mod.rs</strong><dd><code>Register result_schema HTTP
route</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-02556e3f515441b5d2b6bbea7829df488beac03c4cc41309a6638ae3c0e0aec6">+1/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Documentation</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>search.rs</strong><dd><code>Add ResultSchemaResponse
contract</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+7/-0</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Bug
fix</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>TableRenderer.vue</strong><dd><code>Render null/undefined as
empty string</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-794706ea9da3a70012a0327c0464f9fd16802f78ece26055fe951716c2d73d59">+1/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Formatting</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>sqlUtils.ts</strong><dd><code>Minor formatting
fix</code>
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-dde40e8fcb5c37b06c011ede08922c44a3c025682fbc341aee97ba046b73a0e3">+1/-1</a>
</td>
</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>11 files</summary><table>
<tr>
<td><strong>playwright.yml</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-7afcd2d8f7b49bda74843f209eefb7b2da45f7e7803bf2e4bd636699b76aa2d3">+1/-1</a>
</td>
</tr>
<tr>
<td><strong>visualise.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-c14ac1db3e4aa86190d7525e8e38c410bcbbae7f46b01890e74937cc11be67e6">+1/-5</a>
</td>
</tr>
<tr>
<td><strong>logsPage.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-8045b343491d8943869c1af74cdafab204dc49c02a70294417e73f47d35e0b59">+2/-2</a>
</td>
</tr>
<tr>
<td><strong>visualize.spec.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-d144f23552ec150db375c89e52c3a53ede9ff3fa5dcd6092694fbe0a308d22ec">+225/-183</a></td>
</tr>
<tr>
<td><strong>useDashboardPanel.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-fbba1700750053b9eaf38ddd30dab84531bc44131c795ebb713811e4930cd7cb">+420/-6</a>
</td>
</tr>
<tr>
<td><strong>useLogs.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+151/-19</a></td>
</tr>
<tr>
<td><strong>useNotifications.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-32ff5072df9badfebc0a72c401fe337de3324f76b186a52d202eb6cc2fa71e3a">+25/-0</a>
</td>
</tr>
<tr>
<td><strong>en.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+2/-0</a>
</td>
</tr>
<tr>
<td><strong>search.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+43/-0</a>
</td>
</tr>
<tr>
<td><strong>logs.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-637857fa918f47ffb02ebfc8a35b124001ef4f65fd180c467f0f153382d2dc76">+1/-0</a>
</td>
</tr>
<tr>
<td><strong>convertDataIntoUnitValue.ts</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7456/files#diff-14c6fe442ef231566ec411f9e0262cdd24ec912fc67b9154eec743282ab7211a">+120/-29</a></td>
</tr>
</table></details></td></tr></tr></tbody></table>
</details>
___
---------
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: ktx-akshay <akshay.patil@kiara.tech>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: omkark06 <omkar@zinclabs.io>
Co-authored-by: loaki07 <rustyloaki@gmail.com>
Co-authored-by: Loakesh Indiran <66478092+Loaki07@users.noreply.github.com>
Co-authored-by: Bhargav <BJP232004@GMAIL.COM>
Co-authored-by: ktx-vaidehi <vaidehi.akhani@kiara.tech>
Co-authored-by: ktx-vaidehi <134508096+ktx-vaidehi@users.noreply.github.com>
Co-authored-by: Yashodhan Joshi <yashodhan@openobserve.ai>
|
||
|
Huaijin
|
ee7fc8be68 | refactor: datafusion session create and parquet table create (#7993) | ||
|
Huaijin
|
96fdd6b4f8 | fix: super cluster scan_stats (#7989) | ||
|
Yashodhan JoshiandYashodhan Joshi
|
4b3826bbd8 |
fix: add 7d and 13d to no ingestion reporting as well (#7983)
### **User description**
As per requirements, we also want to send events for 7d and 13d no
ingestion along with 24h
___
### **PR Type**
Enhancement, Bug fix
___
### **Description**
- Add 7d and 13d no-ingestion checks
- Generalize job to parameterized durations
- Include duration field in telemetry payload
- Improve logs with duration context
___
### Diagram Walkthrough
```mermaid
flowchart LR
start["start()"]
runner["run_no_ingestion() loop hourly"]
check24["report_org_no_ingestion(24, '24hr')"]
check7d["report_org_no_ingestion(168, '7d')"]
check13d["report_org_no_ingestion(312, '13d')"]
list["organizations::list with created window"]
streams["get_streams(org)"]
segment["report_no_ingestion_to_segment(org, duration)"]
start -- "spawn" --> runner
runner --> check24
runner --> check7d
runner --> check13d
check24 --> list
check7d --> list
check13d --> list
list --> streams
streams -- "no streams" --> segment
```
<details> <summary><h3> File Walkthrough</h3></summary>
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
<td>
<details>
<summary><strong>cloud.rs</strong><dd><code>Parameterize and extend
no-ingestion reporting intervals</code> </dd></summary>
<hr>
src/job/cloud.rs
<ul><li>Replace single 24h job with scheduler loop.<br> <li> Add
parameterized <code>report_org_no_ingestion(start_hour,
duration)</code>.<br> <li> Extend telemetry to include
<code>duration</code> and updated log messages.<br> <li> Add 7d and 13d
reporting alongside 24hr.</ul>
</details>
</td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7983/files#diff-7fc16d008216c106d63f78f75c867fbefe7102433fcaa8893c314f0b7ea8d0e6">+44/-32</a>
</td>
</tr>
</table></td></tr></tr></tbody></table>
</details>
___
Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
|
||
|
Hengfei Yang
|
0869daeb48 | perf: set cache file_list to background (#7970) |