openobserve

starred/openobserve

Fork 0

mirror of https://github.com/openobserve/openobserve.git synced 2026-11-06 00:09:22 +00:00

Files

T

History

udevandhengfei yang 59542d8dd7 fix: remote wal writer improvements (#7108 )

### **PR Type**
Bug fix, Enhancement, Documentation


___

### **Description**
- Expose header size from `WalWriter::new` call

- Adapt ingester to new WAL signature

- Add pipeline WAL writer metrics

- Add task-bound processing warning comment


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>metrics.rs</strong><dd><code>Add pipeline WAL writer
metrics</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/config/src/metrics.rs

<li>Introduce pipeline WAL writer metrics<br> <li> Register new metrics
in <code>register_metrics</code><br> <li> Make
<code>create_const_labels</code> function public


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7108/files#diff-950d37bdd77aa533539acd818a3fcdebd90ccb1f09acb5490b9c0f9b51cb2090">+50/-1</a>&nbsp;
&nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Bug fix</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>writer.rs</strong><dd><code>Adapt ingester writer to
WAL signature change</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/ingester/src/writer.rs

<li>Destructure <code>(WalWriter, usize)</code> tuple from
<code>WalWriter::new</code><br> <li> Use <code>.0</code> to extract the
WAL writer instance


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7108/files#diff-d1a9b1542cc2758afe4fa67abd454dd818718f4e1c7570303519af5a58d6301f">+3/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>writer.rs</strong><dd><code>Return header size from WAL
builder</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/wal/src/writer.rs

<li>Change <code>Writer::new</code> to return <code>(Self,
usize)</code><br> <li> Compute and return <code>header_len</code>
alongside writer<br> <li> Update return signature and path conversion


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7108/files#diff-8821fda229e1219ff38091157f2065914187d34c29ccc8802c374af500f43c1f">+15/-11</a>&nbsp;
</td>

</tr>
</table></td></tr><tr><td><strong>Documentation</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>batch_execution.rs</strong><dd><code>Add processing
task warning</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/service/pipeline/batch_execution.rs

- Add warning comment about task-bound node processing


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7108/files#diff-3f6824a6eddaf1603e5cb0f1a4311ac6a45e1ffb967266ccc1f0bba944625e69">+2/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>

---------

Co-authored-by: hengfei yang <hengfei.yang@gmail.com>

2025-06-11 20:44:51 +05:30

cipher

ci: update rust to nightly-2025-03-02 (#6145 )

2025-03-02 17:27:10 +08:00

cli

ci: improve unit test coverage (#7068 )

2025-06-06 20:37:41 +08:00

common

feat: loki ingest api closes #7034 (#7075 )

2025-06-10 11:02:43 +08:00

config

fix: remote wal writer improvements (#7108 )