mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-04 16:09:05 +00:00
1d0490875b
### **User description** Fixes: - https://github.com/openobserve/openobserve/issues/7986 ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add multi-stream search histogram support - Gate histogram alignment via query flag - Remove unused search_type plumbing - Propagate multi-stream flag through services ___ ### Diagram Walkthrough ```mermaid flowchart LR HTTP["HTTP handlers"] GRPC["gRPC handlers"] Utils["HTTP query utils"] SQL["SQL utils: histogram eligibility"] Hist["SQL: histogram query builder"] Svc["Search service"] Cache["Search cache"] Stream["Streaming pipeline"] Utils -- "parse is_multi_stream_search / enable_align_histogram" --> HTTP Utils -- "parse flags" --> GRPC HTTP -- "pass flags" --> Svc GRPC -- "pass flags" --> Svc Svc -- "eligibility check (multi/unified)" --> SQL Svc -- "partitioning uses align flag" --> Svc Svc -- "search() with multi-stream flag" --> Cache Cache -- "eligibility(query, multi)" --> SQL HTTP -- "convert_to_histogram_query(..., multi)" --> Hist Stream -- "do_search(..., multi)" --> Cache Hist -- "build single/multi-stream histogram query" --> Svc ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>15 files</summary><table> <tr> <td><strong>http.rs</strong><dd><code>Parse multi-stream and align histogram flags</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+20/-0</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Remove SearchPartitionRequest.search_type field</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+0/-3</a> </td> </tr> <tr> <td><strong>sql.rs</strong><dd><code>Add multi-stream histogram eligibility (UNION ALL)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+134/-38</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Pass multi-stream flag into search service</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire flags; histogram conversion uses multi-stream</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-7</a> </td> </tr> <tr> <td><strong>multi_streams.rs</strong><dd><code>Partition multi: add align flag; remove search_type</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+10/-9</a> </td> </tr> <tr> <td><strong>search_inspector.rs</strong><dd><code>Pass multi-stream flag in search profile</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+1/-0</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Stream API supports multi-stream histogram and flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+9/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Propagate multi-stream flag in traces search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Cache search considers multi-stream eligibility</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+2/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Align partitions via flag; multi partition API updates</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+22/-12</a> </td> </tr> <tr> <td><strong>histogram.rs</strong><dd><code>Build histogram for multi-stream queries (UNION ALL)</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+64/-8</a> </td> </tr> <tr> <td><strong>cache.rs</strong><dd><code>Streaming cache handles multi-stream flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+3/-0</a> </td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Execution path passes multi-stream flag to searches</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+25/-4</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Stream request processing threads multi-stream flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+4/-0</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Add multi-stream histogram query support - Align histogram via query flag - Remove search_type plumbing in partitions - Propagate multi-stream flag end-to-end ___ ### Diagram Walkthrough ```mermaid flowchart LR Http["HTTP handlers"] Grpc["gRPC handlers"] Utils["HTTP query utils"] Front["Web client"] Service["Search service"] Cache["Search cache"] SQLU["SQL utils: histogram eligibility"] Hist["Histogram query builder"] Front -- "is_multi_stream, align flag" --> Http Utils -- "parse is_multi_stream/align" --> Http Http -- "enable_align_histogram" --> Service Http -- "is_multi_stream_search" --> Cache Grpc -- "flags default false" --> Service Cache -- "eligibility(sql, multi)" --> SQLU Http -- "convert_to_histogram_query(..., multi)" --> Hist Service -- "partition(gen interval if enabled)" --> Service ``` <details> <summary><h3> File Walkthrough</h3></summary> <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>19 files</summary><table> <tr> <td><strong>http.rs</strong><dd><code>Parse multi-stream and align-histogram flags</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-3b50a69c1ec2f63f8eebb5f9880e17826463a586768aed3e3e15242833609d6b">+20/-0</a> </td> </tr> <tr> <td><strong>search.rs</strong><dd><code>Remove search_type from partition request</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-373e83a31e5bbc82fba05d7ae274b0dbd624ed748272232cf7eb8b8c3b2f3afc">+0/-4</a> </td> </tr> <tr> <td><strong>sql.rs</strong><dd><code>Histogram eligibility with multi-stream UNION ALL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a0a09c66a20d878ec7683ae222c919fbe092b8001871ade7f455b89573ce0911">+134/-38</a></td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Pass default multi-stream flag to service</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-8dcdb10340635be8bb58d45a815569026877f6fdbb3b69a773a30698b91674b0">+3/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Add flags to HTTP search; align histogram handling</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-96630a0dbb6dd7b2b0248fa325c7c7041ce31340c32826675a4a70772e9d29f6">+11/-7</a> </td> </tr> <tr> <td><strong>multi_streams.rs</strong><dd><code>Partition multi endpoint supports align flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3233cdf1e68cb2455f91457719b9c2f477e735ee16546fdfe111e76768bd054">+10/-9</a> </td> </tr> <tr> <td><strong>search_inspector.rs</strong><dd><code>Forward multi-stream flag in inspector search</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f818db4ca10b411c3559cc2a2659c49a59a45ecde0e4a22f9ef023da0bf4a009">+1/-0</a> </td> </tr> <tr> <td><strong>search_stream.rs</strong><dd><code>Streaming search supports multi-stream histogram</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-642de43ff6b19ae41087a9f1efc02d8b0b873809e0c92cbddfac1a30ef3f1184">+9/-3</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Forward multi-stream flag in traces helpers</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-64694c886c809b6efe449e6eba9fbb33d31ce5a3c49cea1a30006fc1d9522665">+2/-0</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Eligibility check aware of multi-stream</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f07bfacc0501b9c234e64b16c1dd8eb0ae8fcbe231f90c81fed72bcc94946f74">+2/-1</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Align histogram in partition; multi aggregate</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-c3b85ea662a885fbd39797968928b291cb59a58f53a03f7b7dfd7ba8b6a7985a">+22/-12</a> </td> </tr> <tr> <td><strong>histogram.rs</strong><dd><code>Build single/multi-stream histogram SQL</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-a5fc4cd11b3ec569b4cedf278ef0b14f540504abfbb23e29eb896d5b2a6cc2b2">+64/-8</a> </td> </tr> <tr> <td><strong>cache.rs</strong><dd><code>Propagate multi-stream flag in streaming cache</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-55769b7cb67b550daf45363c6d7d569b39d4a6235f6a68c091beb40f16b6511e">+3/-0</a> </td> </tr> <tr> <td><strong>execution.rs</strong><dd><code>Pass multi flag through streaming execution</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-75d432351865224f0b00d7cc819554c2ad5fa1f777e9c645d91a3fd5c1113bc0">+25/-4</a> </td> </tr> <tr> <td><strong>mod.rs</strong><dd><code>Wire multi-stream flag across streaming path</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-f236c6c880ed80233fd264e657d5ab658c16c7368bdc70e20f123862b7214374">+4/-0</a> </td> </tr> <tr> <td><strong>IndexList.vue</strong><dd><code>Use align flag in partition requests</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-08f058f883b659e713e606991a7ea37ae1ff016b7cccef970489ab5840dc3d09">+1/-1</a> </td> </tr> <tr> <td><strong>usePanelDataLoader.ts</strong><dd><code>Toggle align histogram for dashboards</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-5a42e9493a7bcd722b88e61fd38883c4f61001bc9cafa399f65f8eafda3aace9">+3/-1</a> </td> </tr> <tr> <td><strong>useLogs.ts</strong><dd><code>Multi-stream histogram flow and params</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-1040f61354f92711e0e77ed4e9bfa15586d45a94c7e2ab06cc37372068030e7b">+3/-2</a> </td> </tr> <tr> <td><strong>search.ts</strong><dd><code>API params for is_multi_stream and align flag</code> </dd></td> <td><a href="https://github.com/openobserve/openobserve/pull/8003/files#diff-885ab872dbc6e7b20264df815e3ef4727f0abc86ce90d2fb47e4aff572814c75">+7/-5</a> </td> </tr> </table></details></td></tr></tr></tbody></table> </details> ___ --------- Co-authored-by: Bhargav <BJP232004@GMAIL.COM> Co-authored-by: ktx-abhay <abhay.padamani@kiara.tech> Co-authored-by: Shrinath Rao <shnath@openobserve.ai> Co-authored-by: omkark06 <omkar@zinclabs.io> Co-authored-by: ktx-akshay <akshay.patil@kiara.tech>
zinc-observe-ui-automation
#Install cypress in project directory npm install cypress --save-dev
To run Tests from local
npx cypress open
To run Tests from local and push result in cypress cloud
./node_modules/.bin/cypress run --record --key cef585a4-1439-4580-bfce-d62230663b22