mirror of
https://github.com/openobserve/openobserve.git
synced 2026-11-05 16:10:43 +00:00
### **PR Type** Tests, Enhancement ___ ### **Description** - Enable `ZO_RESULT_CACHE_ENABLED` by default - Add helper functions and tests for `generate_tantivy_index` - Add comprehensive Puffin module unit tests - Update coverage script to install `cargo nextest` ___ ### **Changes diagram** ```mermaid flowchart LR Config["Change default ZO_RESULT_CACHE_ENABLED"] TantivyMod["Add generate_tantivy_index tests"] PuffinMod["Add Puffin module tests"] Coverage["Update coverage.sh script"] Config --> TantivyMod TantivyMod --> PuffinMod PuffinMod --> Coverage ``` ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Configuration changes</strong></td><td><table> <tr> <td> <details> <summary><strong>config.rs</strong><dd><code>Default result cache enabled</code> </dd></summary> <hr> src/config/src/config.rs - Changed default of `ZO_RESULT_CACHE_ENABLED` from false to true </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7461/files#diff-9aeee4a74c0520daa40f9b81c7016f5b6b3254375b968dd7598e745630980d62">+1/-1</a> </td> </tr> <tr> <td> <details> <summary><strong>coverage.sh</strong><dd><code>Update coverage script</code> </dd></summary> <hr> coverage.sh - Added check/install for `cargo nextest` in coverage task </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7461/files#diff-2033e8e6b7ebfb5c3fe4039f2613d92e65315436b1fdcd3bbe27708370081383">+1/-0</a> </td> </tr> </table></td></tr><tr><td><strong>Tests</strong></td><td><table> <tr> <td> <details> <summary><strong>mod.rs</strong><dd><code>Add generate_tantivy_index tests</code> </dd></summary> <hr> src/service/tantivy/mod.rs <li>Added <code>create_test_batch</code> and <code>create_test_stream</code> helpers<br> <li> Added multiple async tests for <code>generate_tantivy_index</code> and <br><code>create_tantivy_index</code> </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7461/files#diff-68837d2f393b453002ec2d95f6882cfcaf70d9ef6547be2c9a890e8b0fc3c559">+471/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>mod.rs</strong><dd><code>Fix footer size and add puffin tests</code> </dd></summary> <hr> src/service/tantivy/puffin/mod.rs <li>Updated <code>FOOTER_SIZE</code> to include trailing magic<br> <li> Added unit tests for constants and serialization </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7461/files#diff-2f7f0bac01afc6e37908521d1d33d05fcd650b7a08792062976e1e7465b4a1f9">+278/-1</a> </td> </tr> <tr> <td> <details> <summary><strong>reader.rs</strong><dd><code>Add puffin reader tests</code> </dd></summary> <hr> src/service/tantivy/puffin/reader.rs <li>Added tests for <code>PuffinBytesReader</code> and <code>PuffinFooterBytesReader</code><br> <li> Covered parsing, validation, and error scenarios </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7461/files#diff-93d025368ac2b94462df57eb597f345d59967f0dbfb7745a5a7c60b008b85c16">+383/-0</a> </td> </tr> <tr> <td> <details> <summary><strong>writer.rs</strong><dd><code>Add puffin writer tests</code> </dd></summary> <hr> src/service/tantivy/puffin/writer.rs <li>Added tests for <code>PuffinBytesWriter</code> and <code>PuffinFooterWriter</code><br> <li> Verified blob writing, footer structure, and JSON serialization </details> </td> <td><a href="https://github.com/openobserve/openobserve/pull/7461/files#diff-af51d55696b4d2b7e5f4c0ccd8af73e0b464db0c83114c3882548bee6ada5e1d">+306/-0</a> </td> </tr> </table></td></tr></tr></tbody></table> ___ > <details> <summary> Need help?</summary><li>Type <code>/help how to ...</code> in the comments thread for any questions about PR-Agent usage.</li><li>Check out the <a href="https://qodo-merge-docs.qodo.ai/usage-guide/">documentation</a> for more information.</li></details>
106 lines
2.6 KiB
YAML
106 lines
2.6 KiB
YAML
name: Unit tests
|
|
|
|
on:
|
|
push:
|
|
branches:
|
|
- "main"
|
|
paths-ignore:
|
|
- "**.md"
|
|
pull_request:
|
|
branches:
|
|
- "*"
|
|
paths-ignore:
|
|
- "**.md"
|
|
|
|
env:
|
|
COLUMNS: 150
|
|
ZO_RESULT_CACHE_ENABLED: true
|
|
|
|
jobs:
|
|
unit_tests:
|
|
runs-on:
|
|
labels: ubicloud-standard-8
|
|
|
|
services:
|
|
postgres:
|
|
image: postgres:17.5-alpine3.22
|
|
env:
|
|
POSTGRES_PASSWORD: password
|
|
ports:
|
|
- 5432:5432
|
|
# needed because the postgres container does not provide a healthcheck
|
|
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
|
|
mysql:
|
|
image: mysql:9.3
|
|
env:
|
|
MYSQL_ROOT_PASSWORD: password
|
|
MYSQL_DATABASE: mysql
|
|
ports:
|
|
- 3306:3306
|
|
|
|
steps:
|
|
- name: Clone the current repo
|
|
uses: actions/checkout@v4
|
|
|
|
- name: Setup Rust Toolchain
|
|
uses: dtolnay/rust-toolchain@master
|
|
with:
|
|
toolchain: nightly-2025-03-02
|
|
targets: x86_64-unknown-linux-gnu
|
|
- uses: taiki-e/install-action@v2
|
|
with:
|
|
tool: nextest
|
|
- uses: Swatinem/rust-cache@v2
|
|
with:
|
|
cache-on-failure: true
|
|
- name: Install Protoc
|
|
uses: arduino/setup-protoc@v3
|
|
with:
|
|
version: "21.12"
|
|
|
|
- name: Simulate building frontend code
|
|
run: mkdir -p web/dist && touch web/dist/index.html
|
|
|
|
- name: Print rustc cfg
|
|
run: rustc -C target-cpu=native --print cfg
|
|
|
|
- name: Setup cargo config
|
|
run: cargo --config net.git-fetch-with-cli=true clippy -- -D warnings
|
|
|
|
- name: Run unit tests
|
|
run: |
|
|
# required for all dbs
|
|
export ZO_INTERNAL_GRPC_TOKEN=fake
|
|
export ZO_TEXT_DATA_TYPE=text
|
|
|
|
# setup and run with postgres
|
|
echo "Running with postgres"
|
|
export ZO_META_STORE=postgres
|
|
export ZO_META_POSTGRES_DSN=postgres://postgres:password@localhost:5432/postgres
|
|
./coverage.sh run-cov
|
|
|
|
# cleanup
|
|
rm -rf data
|
|
rm -rf ./json
|
|
rm -rf ./src/infra/data
|
|
|
|
# Run wih sqlite
|
|
echo "Running with sqlite"
|
|
export ZO_META_STORE=sqlite
|
|
export ZO_META_POSTGRES_DSN=""
|
|
./coverage.sh run-cov
|
|
|
|
# cleanup
|
|
rm -rf data
|
|
rm -rf ./json
|
|
rm -rf ./src/infra/data
|
|
|
|
# setup and run with mysql
|
|
echo "Running with mysql"
|
|
export ZO_META_STORE=mysql
|
|
export ZO_META_MYSQL_DSN=mysql://root:password@localhost:3306/mysql
|
|
./coverage.sh run-cov
|
|
|
|
# finally print the coverage data
|
|
./coverage.sh check
|