mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 00:09:35 +00:00
Yashodhan JoshiandYashodhan Joshi
babe3b08fb
### **User description** cherry-pick of https://github.com/openobserve/openobserve/pull/7482 for main ___ ### **PR Type** Enhancement, Tests ___ ### **Description** - Add JSON path extraction utility - Extend cipher UDFs with path argument - Rewrite optimizer to handle paths - Add tests for path-based ciphers ___ ### **Changes diagram** ```mermaid flowchart LR J["get_value_from_path"] --> U["cipher UDFs"] U --> O["optimizer rewrite"] O --> T["path-based cipher tests"] ``` ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>auth_tests.rs</strong><dd><code>Remove `batch_size` default in tests</code> </dd></summary> <hr> src/common/utils/auth_tests.rs - Remove deprecated `batch_size` default </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7483/files#diff-2688cbdc0b009b3f0dd829c69ab86c6a7d0778ab3e65e916a410a76aa6ae6480">+0/-1</a> </td> </tr> </table></td></tr><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>json.rs</strong><dd><code>Implement JSON path extraction utility</code> </dd></summary> <hr> src/config/src/utils/json.rs <li>Import <code>itertools</code> for flattening<br> <li> Add <code>get_value_from_path</code> implementation<br> <li> Handle nested JSON and wildcard paths<br> <li> Add unit tests for JSON path extraction </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7483/files#diff-f638bd6d218ad53a671681c2fa8e420cfbf8369e6e93269c9559e9711338d52a">+131/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>cipher.rs</strong><dd><code>Support path argument in optimizer cipher rewrite</code> </dd></summary> <hr> src/service/search/datafusion/optimizer/cipher.rs <li>Require only two args for rewrite checks<br> <li> Update sanity checks to allow optional path<br> <li> Preserve or default path in <code>CipherKeyRewrite</code><br> <li> Add tests for path-based cipher calls </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7483/files#diff-1dfdd6163a17d67d426ab189446fa4c43551094aea50ffd82d334abd3ffe4f81">+40/-11</a> </td> </tr> <tr> <td> <details> <summary><strong>cipher_udf.rs</strong><dd><code>Extend cipher UDFs with JSON path support</code> </dd></summary> <hr> src/service/search/datafusion/udf/cipher_udf.rs <li>Extend UDFs to accept three parameters<br> <li> Validate third <code>path</code> argument in UDFs<br> <li> Parse JSON and extract value at <code>path</code><br> <li> Update encryption/decryption logic for arrays </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7483/files#diff-aca30595e1e899bf83b213e84d91063cc621a8972ce22e67bcf4705af817188f">+175/-19</a></td> </tr> <tr> <td> <details> <summary><strong>sql.rs</strong><dd><code>Require three arguments in SQL cipher parser</code> </dd></summary> <hr> src/service/search/sql.rs <li>Update cipher function argument count to three<br> <li> Adjust SQL parser validation for paths </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7483/files#diff-d9ccf393c53794b0ef2816eef12742c26e23d2788cde27780af7570662aea446">+1/-1</a> </td> </tr> </table></td></tr></tr></tbody></table> ___ > <details> <summary> Need help?</summary><li>Type <code>/help how to ...</code> in the comments thread for any questions about PR-Agent usage.</li><li>Check out the <a href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a> for more information.</li></details> --------- Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com> Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>