Files
openobserve/.github/workflows/unit-tests.yml
T
Hengfei Yang 3102168008 ci: unit test (#7461)
### **PR Type**
Tests, Enhancement


___

### **Description**
- Enable `ZO_RESULT_CACHE_ENABLED` by default

- Add helper functions and tests for `generate_tantivy_index`

- Add comprehensive Puffin module unit tests

- Update coverage script to install `cargo nextest`


___

### **Changes diagram**

```mermaid
flowchart LR
  Config["Change default ZO_RESULT_CACHE_ENABLED"]
  TantivyMod["Add generate_tantivy_index tests"]
  PuffinMod["Add Puffin module tests"]
  Coverage["Update coverage.sh script"]
  Config --> TantivyMod
  TantivyMod --> PuffinMod
  PuffinMod --> Coverage
```


___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>config.rs</strong><dd><code>Default result cache
enabled</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></summary>
<hr>

src/config/src/config.rs

- Changed default of `ZO_RESULT_CACHE_ENABLED` from false to true


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7461/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>coverage.sh</strong><dd><code>Update coverage
script</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

coverage.sh

- Added check/install for `cargo nextest` in coverage task


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7461/files#diff-2033e8e6b7ebfb5c3fe4039f2613d92e65315436b1fdcd3bbe27708370081383">+1/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Tests</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>mod.rs</strong><dd><code>Add generate_tantivy_index
tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/service/tantivy/mod.rs

<li>Added <code>create_test_batch</code> and
<code>create_test_stream</code> helpers<br> <li> Added multiple async
tests for <code>generate_tantivy_index</code> and
<br><code>create_tantivy_index</code>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7461/files#diff-68837d2f393b453002ec2d95f6882cfcaf70d9ef6547be2c9a890e8b0fc3c559">+471/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mod.rs</strong><dd><code>Fix footer size and add puffin
tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/service/tantivy/puffin/mod.rs

<li>Updated <code>FOOTER_SIZE</code> to include trailing magic<br> <li>
Added unit tests for constants and serialization


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7461/files#diff-2f7f0bac01afc6e37908521d1d33d05fcd650b7a08792062976e1e7465b4a1f9">+278/-1</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>reader.rs</strong><dd><code>Add puffin reader
tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/service/tantivy/puffin/reader.rs

<li>Added tests for <code>PuffinBytesReader</code> and
<code>PuffinFooterBytesReader</code><br> <li> Covered parsing,
validation, and error scenarios


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7461/files#diff-93d025368ac2b94462df57eb597f345d59967f0dbfb7745a5a7c60b008b85c16">+383/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>writer.rs</strong><dd><code>Add puffin writer
tests</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

src/service/tantivy/puffin/writer.rs

<li>Added tests for <code>PuffinBytesWriter</code> and
<code>PuffinFooterWriter</code><br> <li> Verified blob writing, footer
structure, and JSON serialization


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/7461/files#diff-af51d55696b4d2b7e5f4c0ccd8af73e0b464db0c83114c3882548bee6ada5e1d">+306/-0</a>&nbsp;
</td>

</tr>
</table></td></tr></tr></tbody></table>

___

> <details> <summary> Need help?</summary><li>Type <code>/help how to
...</code> in the comments thread for any questions about PR-Agent
usage.</li><li>Check out the <a
href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a>
for more information.</li></details>
2025-07-04 17:40:29 +05:30

106 lines
2.6 KiB
YAML

name: Unit tests
on:
push:
branches:
- "main"
paths-ignore:
- "**.md"
pull_request:
branches:
- "*"
paths-ignore:
- "**.md"
env:
COLUMNS: 150
ZO_RESULT_CACHE_ENABLED: true
jobs:
unit_tests:
runs-on:
labels: ubicloud-standard-8
services:
postgres:
image: postgres:17.5-alpine3.22
env:
POSTGRES_PASSWORD: password
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
mysql:
image: mysql:9.3
env:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: mysql
ports:
- 3306:3306
steps:
- name: Clone the current repo
uses: actions/checkout@v4
- name: Setup Rust Toolchain
uses: dtolnay/rust-toolchain@master
with:
toolchain: nightly-2025-03-02
targets: x86_64-unknown-linux-gnu
- uses: taiki-e/install-action@v2
with:
tool: nextest
- uses: Swatinem/rust-cache@v2
with:
cache-on-failure: true
- name: Install Protoc
uses: arduino/setup-protoc@v3
with:
version: "21.12"
- name: Simulate building frontend code
run: mkdir -p web/dist && touch web/dist/index.html
- name: Print rustc cfg
run: rustc -C target-cpu=native --print cfg
- name: Setup cargo config
run: cargo --config net.git-fetch-with-cli=true clippy -- -D warnings
- name: Run unit tests
run: |
# required for all dbs
export ZO_INTERNAL_GRPC_TOKEN=fake
export ZO_TEXT_DATA_TYPE=text
# setup and run with postgres
echo "Running with postgres"
export ZO_META_STORE=postgres
export ZO_META_POSTGRES_DSN=postgres://postgres:password@localhost:5432/postgres
./coverage.sh run-cov
# cleanup
rm -rf data
rm -rf ./json
rm -rf ./src/infra/data
# Run wih sqlite
echo "Running with sqlite"
export ZO_META_STORE=sqlite
export ZO_META_POSTGRES_DSN=""
./coverage.sh run-cov
# cleanup
rm -rf data
rm -rf ./json
rm -rf ./src/infra/data
# setup and run with mysql
echo "Running with mysql"
export ZO_META_STORE=mysql
export ZO_META_MYSQL_DSN=mysql://root:password@localhost:3306/mysql
./coverage.sh run-cov
# finally print the coverage data
./coverage.sh check