Commit Graph
357 Commits
Author SHA1 Message Date
Hengfei Yang 75bda12998 chore: change default max size to 2GB for compactor (#11629) 2026-05-06 23:07:25 +08:00
dependabot[bot] e6c164ba07 chore(deps): bump rand from 0.10.0 to 0.10.1 (#11260) 2026-04-23 08:23:02 +08:00
Hengfei Yang 77d376efa2 fix: improve real ip logic (#11378) 2026-04-22 22:56:05 +08:00
Huaijin 493dac7433 chore: update datafusion to 53.1.0 (#11310) 2026-04-17 12:03:02 +00:00
Hengfei Yang 69fd00f0c5 fix: implement AES-256-SIV encryption and decryption utilities (#11286) 2026-04-16 19:15:56 +08:00
Hengfei Yang b266560838 feat: support custom AI tools (#11270) 2026-04-15 20:11:35 +08:00
漂流 fbf7283ffc chore: use std instead of once_cell (#11081) 2026-04-06 16:27:16 +08:00
HuaijinandHengfei Yang ada8b3763f chore: upgrade datafusion to 53 (#11093)
- [x] check if still need `CoalesceBatchesExec`
- [x] check object_store upgrade

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2026-04-06 06:06:51 +00:00
udev 9525aad6fe fix: grpc use native/webpki trust store (#11083) 2026-03-31 11:28:53 +05:30
477ec39db2 feat: add support for sourcemaps for RUM (#10259)
This ads support for sourcemaps in RUM

Design at :
https://github.com/openobserve/designs/blob/main/application-monitoring/rum/sourcemap-be.md

---------

Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: sai nikhil kethe <nikhil@openobserve.ai>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-03-24 21:18:06 +05:30
Huaijin 2923938b95 chore: upgrade datafusion to 52.4.0 (#10995) 2026-03-23 16:47:21 +08:00
Huaijin b3a9b3c14f perf: preserve fetch when rewriter FilterExec (#10971) 2026-03-21 20:15:35 +08:00
Hengfei Yang b50de58a97 refactor: update schema field naming and enhance metadata extraction methods (#10945) 2026-03-18 09:01:31 +08:00
Hengfei Yang 53668c8c1a chore: update dependencies in Cargo files (#10929) 2026-03-17 11:14:14 +05:30
dependabot[bot] 283a5f3041 chore(deps): bump lz4_flex from 0.11.5 to 0.11.6 (#10928) 2026-03-17 09:08:02 +08:00
Hengfei Yang dafa382abb fix: handle serde_json arbitrary_precision number in VRL parse_json (#10920) 2026-03-16 21:07:35 +08:00
Hengfei Yang 5d4a909813 fix: create trigger schema (#10896) 2026-03-13 22:36:41 +08:00
Sai Nikhil d2861433fa fix: search inspector UI (#10749) 2026-03-13 09:57:29 +08:00
Huaijin 8212ca6f8e refactor: split the tantivy out of core (#10825)
perpare for tantivy benchmark
2026-03-10 17:16:45 +08:00
Hengfei Yang 0b81010cf1 ci: update mimalloc version (#10756) 2026-03-05 16:10:11 +08:00
Hengfei Yang 020c41e913 fix: WAL runtime worker configuration (#10753) 2026-03-05 11:55:24 +08:00
Hengfei Yang 289d806598 feat: add support for dynamic tracing resource attributes (#10731) 2026-03-04 20:29:18 +08:00
Harsh MahajanandHarsh Mahajan ef369e595d chore: upgrading packaages (only non-core packages) (#10726)
Co-authored-by: Harsh Mahajan <undefined@users.noreply.github.com>
2026-03-04 05:45:36 +05:30
Huaijin 9944f02031 chore: upgrade datafusion to 52.2.0 and vortex to 0.60.0 (#10706) 2026-03-02 16:06:21 +08:00
dependabot[bot] d79703326b chore(deps): bump keccak from 0.1.5 to 0.1.6 (#10591) 2026-02-23 11:33:20 +08:00
Huaijin a810617abc chore: refacotr vortex related code (#10506) 2026-02-13 16:42:40 +08:00
Huaijin 016d1d777c feat: support vortex as option file format (#10491) 2026-02-12 19:16:51 +08:00
udev 9b11d0346a fix: JWT dependency (#10437)
### **PR Type**
Bug fix, Dependencies


___

### **Description**
- Enable `aws_lc_rs` for `jsonwebtoken`

- Fix JWT crypto backend compatibility


___

### Diagram Walkthrough


```mermaid
flowchart LR
  dep["`jsonwebtoken` dependency"] 
  feat["Enable `aws_lc_rs` feature"]
  jwt["JWT signing/verification compatibility"]
  dep -- "adds feature flag" --> feat
  feat -- "ensures backend support" --> jwt
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Dependencies</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>Cargo.toml</strong><dd><code>Enable AWS LC backend for
`jsonwebtoken`</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

Cargo.toml

<ul><li>Switch <code>jsonwebtoken</code> to feature-enabled
dependency<br> <li> Enable <code>aws_lc_rs</code> feature for
<code>jsonwebtoken</code> v10.3</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/10437/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

</details>

___
2026-02-10 10:58:20 +05:30
Huaijin b315f4ca08 feat: dag graph for llm traces (#10388) 2026-02-06 16:46:44 +08:00
Harsh Mahajan e6d5b6eed5 fix: ParserError on streams with : in name (#10394) 2026-02-06 12:46:49 +08:00
Harsh Mahajan d0b0614919 fix: dependency upgrades in Cargo.toml and Cargo.lock. (#10253) 2026-02-06 11:45:24 +08:00
Huaijin 21155591fd feat: map otel field to openobserve field (#10213) 2026-02-04 15:30:55 +08:00
Harsh Mahajan 97056c0777 fix: cargo audit, package upgrade (#10363) 2026-02-04 09:36:28 +08:00
Huaijin 19034ca3a4 feat: support memory profiling with jemalloc and cpu profiling (#10317) 2026-02-02 16:37:16 +08:00
dependabot[bot] cef79a7419 chore(deps): bump oneshot from 0.1.8 to 0.1.13 (#10191) 2026-02-01 22:46:03 +08:00
Hengfei Yang fac4b9eecc fix: result cache delta end time to fix duplicates during delta search (#10313) 2026-02-01 22:42:13 +08:00
Hengfei Yang 6e18a14e6f fix: delete stream when data retention disabled (#10263) 2026-01-30 23:12:10 +08:00
Huaijin 7672bc12f3 chore: upgrade datafusion to 52 (#10090) 2026-01-19 19:59:48 +08:00
Hengfei Yang fdcb375256 fix: unit tests (#10032) 2026-01-14 20:32:41 +08:00
Hengfei Yang 1fc4270f54 fix: payload limit (#10021)
1. We only need `ZO_PAYLOAD_LIMIT` now, removed  `ZO_JSON_LIMIT`
2. Config axum log level `RUST_LOG = "info,config::axum=warn"`
2026-01-14 16:54:35 +08:00
Hengfei Yang 33eecaefe3 refactor: migrate actix-web to axum (#9912) 2026-01-13 16:44:53 +08:00
Ashish KolheandClaude Opus 4.5 ce09cd0a4d feat: add js for claim parser (#9873)
### **User description**
Design at : security/sso-claim-parser/configuration.md


___

### **PR Type**
Enhancement, Tests


___

### **Description**
- Introduce rquickjs-based JS runtime for functions

- Add compile_js_function and apply_js_fn utilities

- Extend pipeline to support JS via CompiledFunctionRuntime

- Update web UI: select JS/VRL transforms, show JS errors


___

### Diagram Walkthrough


```mermaid
flowchart LR
  A["FunctionNode"] -->|JS| B["compile_js_function"]
  A -->|VRL| C["compile_vrl_function"]
  B --> D["apply_js_fn"]
  C --> E["apply_vrl_fn"]
  D --> F["send to children"]
  E --> F
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>13
files</summary><table>
<tr>
<td><strong>function.ts</strong><dd><code>Add optional `trans_type` to
TestFunctionPayload</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-c6766f6c671adce05f9eef5d4a57a32371fa7da35628ef5c142fa503d191cb16">+16/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>js.rs</strong><dd><code>Introduce JS runtime, compile and
apply functions</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-88e490bed5fa6c9c8e0aa2738b95c5284bb97899298f54b9cd587713e3ef0109">+756/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>batch_execution.rs</strong><dd><code>Extend pipeline
execution to handle JS functions</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-3f6824a6eddaf1603e5cb0f1a4311ac6a45e1ffb967266ccc1f0bba944625e69">+465/-96</a></td>

</tr>

<tr>
<td><strong>functions.rs</strong><dd><code>Validate and test-run JS
transforms, restrict to _meta</code>&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-ed5611a9aa6b5c033e70d7631ba72fafad3cc842672077a7fcf965d11a4fef23">+319/-35</a></td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Forbid JavaScript functions in
pipelines globally</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-c186c796a7bdfd9d1301b1429df11456430f1d0d4758783810b9e2e44384e3b2">+39/-1</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>jwt.rs</strong><dd><code>Add JS claim parser execution
function</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-232cab74f4cbf135af424744272e17732fbda8ac005d33c0acb4597e65479d02">+25/-2</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Wrap JS compile/apply in ingestion
API</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-7c7b5bca85f0ced652965cae66540c96f56dfcca4f4c16b77bf7b45aac203d1b">+24/-1</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Include `trans_type` in
test_function API payload</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-a5e1b3644cef9feb689dfe66382e44e5434ecadbff3469127346a333d4733102">+8/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>AddFunction.vue</strong><dd><code>UI: add JS/VRL transform
type selector</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-ebfd6c538f3cd3ecb9ab17550fd76e2e89e7a1abccc83a7505077e4af3eee110">+42/-25</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>FunctionsToolbar.vue</strong><dd><code>Toolbar: transform
type radio buttons</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-8283d2ff9c4a66b4b59eac306672aca2d54655a462e2ddb88e14e039d79dcdaf">+51/-3</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>DomainManagement.vue</strong><dd><code>Track claim-parser
function changes for save button</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-5e47cd01fcb63f7ef27dc64d2ecf66c39e17097e29944951c3ae1bfe715bb97c">+17/-9</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>TestFunction.vue</strong><dd><code>Support JS error
messaging in test UI</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-b9af9a2d1bf04d23422c26a13e6a763c6953c8713357a2f64ed8073ce6c926a3">+15/-5</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>PipelineEditor.vue</strong><dd><code>Filter out JS functions
in pipeline editor UI</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-68c21f647336067eb79737ce9aca2d49632f2fa1c0878bcfdae55607e3473a15">+12/-4</a>&nbsp;
&nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>2 files</summary><table>
<tr>
<td><strong>function.rs</strong><dd><code>Extend `Transform` model for
JS support</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-27c9f3d93723397a0e234bc45c9f543b43eff5fd518d76e735f78d95a58f3f1c">+16/-6</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Register new `js` utils
module</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-2aa59ce4464e7dbabdb9367fc8bb8c1b489c914b8690577d4916870463ab12db">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Bug
fix</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>errors.rs</strong><dd><code>Rename
`MemoryTableOverflowError` display message</code>&nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-bf45932466265b732f13d0cd9b9cffbaefbe37c5861ace9315449eeca9250713">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>3
files</summary><table>
<tr>
<td><strong>result-array-functions.spec.js</strong><dd><code>Add E2E
tests for `#ResultArray#` functions in UI</code>&nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-e9911625ad2bab76f2b760f0ad254791331993a6a874b96f6a11fc6d88038b10">+470/-0</a>&nbsp;
</td>

</tr>

<tr>
<td><strong>cleanup.spec.js</strong><dd><code>Clean up JS function test
data in UI tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-c214275a512fa285a7e4f2ab61dfcb42d71a0436b0e4241f94d7b45693587983">+31/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>page-manager.js</strong><dd><code>Register `FunctionsPage`
in `PageManager`</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-565eab8b3c767ae50c9d05e591621d14f66b81af84d6b5ea6efe136c1c3933df">+6/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

</table></details></td></tr><tr><td><strong>Dependencies</strong></td><td><details><summary>1
files</summary><table>
<tr>
<td><strong>Cargo.toml</strong><dd><code>Add `rquickjs` dependency for
JS runtime</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Additional
files</strong></td><td><details><summary>18 files</summary><table>
<tr>
  <td><strong>test_functions_meta_org_restriction.py</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-99ef1207d6ae6d7eeaed37d70c8cf0c4770ba78f6f51fbdc067d35492d2fe9ea">+401/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>test_js_enrichment.py</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-c5547d46bec60b888d947d1e9b7c5445aee72cf8e837182f7138e0fc903eedbb">+356/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>test_js_result_array.py</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-9ed1363a1f58c94e018bb2b0aaa9d08fa71f89e7e55a6ff6be4a82bbeed8985c">+390/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>functionsPage.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-7bab9cc2eb8f36d3ee5d0aab453c353ec6d6dfecbc401cf401322ff57faa8440">+326/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>javascript-functions.spec.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-bdae6529346a7222bdec19f75f490cb9d0f3578a260ba835225efcd27db3f64e">+314/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>meta-org-js-restriction.spec.js</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-16cc2373f6f1955f8344a59d5f2edcd6ce121c4fdee1d432b1f0ec8e27f25881">+483/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>FunctionList.vue</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-6ff9af50a1597504b7f0dca91e61cf30c800b4a17b4be2ed2c3a4f7a7b2d7068">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td><strong>de.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-78f24ecb2dbcfc38fdf3ec22b2559a5b40d236ea689d1c55f5d3e87168310264">+30/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>en.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-892aa3d75ab6e129ef9116aa21bec1b9a15ba69ce5366d2a19128078c690d824">+15/-7</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td><strong>es.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-ac2a2167b8b973e3f776f9fe3b43bef4576470d8ad33f3d7cfaff6edff0e5778">+30/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>fr.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-db80e762b88ddf183820c050cdc6b215e6b02b365b5e73f5152af46b0d26f1a2">+30/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>it.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-2de78074a5f887afddce60d694ead9322e6ab223eb70251c13819f3028bdc70d">+30/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>ja.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-092e49c5f9fbc1b539ab9c2866d108d0ce1d314022b4c294eb4611b285e7bbd3">+30/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>ko.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-50e27ca1b4825a5026f3a251bfcf79c97cab07cfade2ebd512d332453d707c48">+30/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>nl.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-a1706021687ae5d8a1c006c49eab3946174fecd895f9b70a89c2e76d97250906">+30/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>pt.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-38e804f64cb5e0e451849fc84e9b46e53c954a4a4263d51c18665ad45f4b4815">+30/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>tr.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-6f0f131e9e2591480c4c5d1eaffc72d02dc31cbd8d3f90d8f012f08cabac8818">+30/-18</a>&nbsp;
</td>

</tr>

<tr>
  <td><strong>zh.json</strong></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9873/files#diff-8818055db58f83230aeea4ebfd4cec31d924e8c47b6f91298e356959ab25339e">+254/-242</a></td>

</tr>
</table></details></td></tr></tr></tbody></table>

</details>

___

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 22:47:35 +05:30
dependabot[bot] b90334fa27 chore(deps): bump rsa from 0.9.9 to 0.9.10 (#9867) 2026-01-07 11:34:32 +08:00
Hengfei Yang c96605a093 fix: metrics streaming partition (#9839) 2026-01-06 01:07:10 +08:00
Hengfei Yang 0e4228dcff perf: add runtime module for metrics (#9813) 2026-01-02 10:46:19 +08:00
Subhra264andHengfei Yang 6cc7caf6c3 fix: only origin region ingesters should process the enrichment table url job (#9782)
### **User description**
Currently, when the enrichment table url job is created, it is only
processed by the ingesters of the same origin region which is correct.
But, the background job responsible for reprocessing the stale jobs can
process enrichment table url from other regions as well which is not
ideal. This pr, adds a is_local_region field to the
enrichment_table_urls table to indicate whether the db record was
created in the same region. The background job in ingesters to process
stale jobs, hence, now only process the jobs created in that region.

This change applies to only the background jobs that processes stale
jobs to ensure enrichment table data is present only in the origin
region. The actual job processing (when the user creates the job)
happens through router and hence is always processed by the ingesters of
origin region.


___

### **PR Type**
Bug fix, Enhancement


___

### **Description**
- Add `is_local_region` flag to URL jobs

- Filter stale-job recovery by `is_local_region`

- Initialize URL processor on ingester startup

- Mark remote jobs with `is_local_region=false`


___

### Diagram Walkthrough


```mermaid
flowchart LR
  JC["URL Job Created\n(is_local_region=true)"]
  RC["Remote Job Received\n(is_local_region=false)"]
  DB["enrichment_table_urls\ndatabase record"]
  SJ["Stale Job Recovery\n(filters is_local_region=true)"]
  JC -- "save_url_job" --> DB
  RC -- "super_cluster_queue.put" --> DB
  SJ -- "claim_stale_jobs" --> DB
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>6
files</summary><table>
<tr>
<td><strong>enrichment_table_urls.rs</strong><dd><code>Add and propagate
`is_local_region` field</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9782/files#diff-e9ba1ba6e742e8d7652c3f31765de43bee061e7983bca67ffefecf8590210188">+11/-6</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>enrichment_table_urls.rs</strong><dd><code>Include
`is_local_region` in ORM model</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9782/files#diff-9c3081476927bd337b2cbed8310578869845a6137a2c5305f677affb7604d39f">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Initialize URL processor at
startup</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9782/files#diff-f2e4c29b8eef22189ec704a832633c4bd135a24444f054864c98064bd0963681">+7/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>enrichment_table.rs</strong><dd><code>Set
`is_local_region=true` on new jobs</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9782/files#diff-6ce27ddf8a0d8b4e790bda90d088ff692a5da31a8d330beea8a9dd843294572b">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Re-export `init_url_processor`
function</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9782/files#diff-3e2b6d3b8da1f9e73aca279052a97e683184d68256c23215b18ffee885836bbf">+3/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
<td><strong>url_processor.rs</strong><dd><code>Add `init_url_processor`
for recovery task</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9782/files#diff-d39589e34157fe7930563b9bc5740128e168b8d8ac0a90371a4bdf1e52ce058b">+39/-0</a>&nbsp;
&nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>2 files</summary><table>
<tr>

<td><strong>m20251226_000001_add_enrichment_table_urls_is_local_region.rs</strong><dd><code>Migration:
add `is_local_region` column</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9782/files#diff-25f7d4220c44499608e8b139ba530335a1426f81988fae2e48d1b41aaacdfb05">+80/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
<td><strong>mod.rs</strong><dd><code>Register new region-flag
migration</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9782/files#diff-7ee42a714d3d56c663da0412d15514613222055aebb5b50dafbc051a656a36f6">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr><tr><td><strong>Bug
fix</strong></td><td><details><summary>1 files</summary><table>
<tr>
<td><strong>pipelines.rs</strong><dd><code>Mark remote jobs
`is_local_region=false`</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9782/files#diff-7c8aa678045dd00de89d4c2345cf4a0511ce9bfac5d6f64e7404178c81e541ad">+3/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr></tr></tbody></table>

</details>

___

---------

Co-authored-by: Hengfei Yang <hengfei.yang@gmail.com>
2025-12-29 13:12:51 +05:30
Hengfei Yang 511f2389c6 feat: new db migration (#9774) 2025-12-27 21:05:07 +08:00
Hengfei Yang 1714f6c570 fix: metrics stream can't be deleted (#9710) 2025-12-23 14:49:04 +08:00
Yashodhan Joshi 330a179d1c refactor: file list dump and stream stats (#9236) 2025-12-17 09:44:42 +08:00
udev 99c3391575 fix: audit (#9417)
This brings changes to the MMDB parser logic. Need to test via VRL
functions.
2025-12-03 15:33:04 +05:30