Files
openobserve/.github/workflows/unit-tests.yml
T
Yashodhan JoshiandYashodhan Joshi 8f4cdbb6b5 fix: remove arduino protoc action (#9725)
### **User description**
We have recently observed that a lot of times the install protoc action
errors, so this instead removes the dependency on action and uses the
release url directly to download protoc. Technically it still goes
through the same github api, but now we have more precise control on
what is being called.

The next best thing would be vendoring the zip itself in the repo, which
I'm not particularly happy with, but can be tried.


___

### **PR Type**
Bug fix, Enhancement


___

### **Description**
- Add manual protoc installation script

- Replace setup-protoc action in workflows

- Support x86_64 and aarch64 detection

- Error handling for unsupported platforms


___

### Diagram Walkthrough


```mermaid
flowchart LR
  CI["CI Workflow"] -- "execute protoc.sh" --> Script["protoc.sh"]
  Script -- "download & unzip protoc" --> Setup["Protoc setup steps"]
  Setup -- "copy binaries & headers" --> Done["Protoc installed"]
```



<details> <summary><h3> File Walkthrough</h3></summary>

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>protoc.sh</strong><dd><code>Add manual protoc
installation script</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

.github/protoc.sh

<ul><li>Add shell script for manual protoc install<br> <li> Detect
platform (x86_64, aarch64)<br> <li> Download and unzip specified
release<br> <li> Copy binary and include files system-wide</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9725/files#diff-c91d907e5a707c73726a6c869b8391f8bed703cb1cbe9b54fc81fdef67a3e804">+19/-0</a>&nbsp;
&nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Configuration
changes</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>api-testing.yml</strong><dd><code>Replace setup-protoc
in API testing CI</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

.github/workflows/api-testing.yml

<ul><li>Remove arduino/setup-protoc action<br> <li> Add run step
invoking protoc.sh</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9725/files#diff-bf87f46624aaa9c4f1d9faa033baf531a543f2d4ac188cd64e9ab3febfee8f9e">+2/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>build-pr-image.yml</strong><dd><code>Replace
setup-protoc in build-pr-image CI</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

.github/workflows/build-pr-image.yml

- Replace setup-protoc action
- Invoke protoc.sh in build-pr-image CI


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9725/files#diff-ebeabab3ec6b3ad90cc66cb9c0d9f2ad72c7ec6116e30e9801241b46992742d6">+8/-6</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>db-testing.yml</strong><dd><code>Replace setup-protoc
in DB testing CI</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

.github/workflows/db-testing.yml

- Remove setup-protoc action
- Invoke protoc.sh in DB testing CI


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9725/files#diff-25d80f07432f01a3a288495d1d29a8457c2897d5ec819137dea15ad239b122f9">+3/-4</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>playwright.yml</strong><dd><code>Replace setup-protoc
in Playwright CI</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

.github/workflows/playwright.yml

- Replace setup-protoc action
- Execute protoc.sh in Playwright CI


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9725/files#diff-7afcd2d8f7b49bda74843f209eefb7b2da45f7e7803bf2e4bd636699b76aa2d3">+4/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>unit-tests.yml</strong><dd><code>Replace setup-protoc
in unit tests CI</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

.github/workflows/unit-tests.yml

<ul><li>Remove setup-protoc action in unit tests<br> <li> Call protoc.sh
when changes detected</ul>


</details>


  </td>
<td><a
href="https://github.com/openobserve/openobserve/pull/9725/files#diff-cd0effce06f425599e96952f4d5f684641a683f46d9909e7c74a9f8b2f03ccd3">+3/-3</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

</details>

___

---------

Signed-off-by: Yashodhan Joshi <yjdoc2@gmail.com>
Co-authored-by: Yashodhan Joshi <yjdoc2@gmail.com>
2025-12-24 14:16:51 +05:30

184 lines
5.4 KiB
YAML

name: Unit tests
on:
push:
branches:
- "main"
# paths-ignore:
# - "**.md"
# - "**.yml"
# - "**.yaml"
pull_request:
branches:
- "*"
# paths-ignore:
# - "**.md"
# - "**.yml"
# - "**.yaml"
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
env:
COLUMNS: 150
ZO_RESULT_CACHE_ENABLED: true
jobs:
backend_unit_tests:
runs-on:
labels: ubicloud-standard-8
permissions:
contents: read
# Note: Services always start regardless of conditional logic in steps.
# GitHub Actions does not support conditional services.
services:
postgres:
image: postgres:17.5-alpine3.22
env:
POSTGRES_PASSWORD: password
ports:
- 5432:5432
# needed because the postgres container does not provide a healthcheck
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
steps:
- name: Clone the current repo
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Check if API code has changes
id: check_changes
run: |
if git diff --name-only origin/${{ github.base_ref || 'main' }} ${{ github.sha }} | grep -qE '^src/|^Cargo\.(toml|lock)$'; then
echo "has_changes=true" >> $GITHUB_OUTPUT
else
echo "has_changes=false" >> $GITHUB_OUTPUT
fi
- name: Setup Rust Toolchain
if: steps.check_changes.outputs.has_changes == 'true'
uses: dtolnay/rust-toolchain@master
with:
toolchain: nightly-2025-11-11
targets: x86_64-unknown-linux-gnu
- uses: taiki-e/install-action@v2
if: steps.check_changes.outputs.has_changes == 'true'
with:
tool: nextest
- uses: Swatinem/rust-cache@v2
if: steps.check_changes.outputs.has_changes == 'true'
with:
cache-on-failure: true
- name: Install Protoc
if: steps.check_changes.outputs.has_changes == 'true'
run: |
bash ./.github/protoc.sh
- name: Simulate building frontend code
if: steps.check_changes.outputs.has_changes == 'true'
run: mkdir -p web/dist && touch web/dist/index.html
- name: Print rustc cfg
if: steps.check_changes.outputs.has_changes == 'true'
run: rustc -C target-cpu=native --print cfg
- name: Check code formatting
if: steps.check_changes.outputs.has_changes == 'true'
run: cargo fmt --all -- --check
- name: Run clippy
if: steps.check_changes.outputs.has_changes == 'true'
run: cargo --config net.git-fetch-with-cli=true clippy -- -D warnings
- name: Run unit tests
if: steps.check_changes.outputs.has_changes == 'true'
run: |
# required for all dbs
export ZO_INTERNAL_GRPC_TOKEN=fake
export ZO_TEXT_DATA_TYPE=text
# setup and run with postgres
echo "Running with postgres"
export ZO_META_STORE=postgres
export ZO_META_POSTGRES_DSN=postgres://postgres:password@localhost:5432/postgres
./coverage.sh run-cov
# cleanup
rm -rf data ./json ./src/infra/data
# Run with sqlite
echo "Running with sqlite"
export ZO_META_STORE=sqlite
export ZO_META_POSTGRES_DSN=""
./coverage.sh run-cov
# finally check the coverage data
./coverage.sh check
ui_unit_tests:
runs-on:
labels: ubicloud-standard-8
permissions:
contents: read
steps:
- name: Clone the current repo
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: Check if frontend code has changes
id: check_changes
run: |
if git diff --name-only origin/${{ github.base_ref || 'main' }} ${{ github.sha }} | grep -q "^web/"; then
echo "has_changes=true" >> $GITHUB_OUTPUT
else
echo "has_changes=false" >> $GITHUB_OUTPUT
fi
- name: Setup Node.js
if: steps.check_changes.outputs.has_changes == 'true'
uses: actions/setup-node@v5
with:
node-version: '22'
cache: 'npm'
cache-dependency-path: web/package-lock.json
- name: Install dependencies
if: steps.check_changes.outputs.has_changes == 'true'
working-directory: web
run: npm ci
- name: Run frontend unit tests with coverage and check thresholds
if: steps.check_changes.outputs.has_changes == 'true'
working-directory: web
run: ./coverage.sh check
unit_tests_summary:
runs-on: ubuntu-latest
permissions: {}
needs: [backend_unit_tests, ui_unit_tests]
if: always()
steps:
- name: Check test results
run: |
if [ "${{ needs.backend_unit_tests.result }}" == "success" ] || [ "${{ needs.backend_unit_tests.result }}" == "skipped" ]; then
echo "Backend tests passed or skipped"
else
echo "Backend tests failed"
exit 1
fi
if [ "${{ needs.ui_unit_tests.result }}" == "success" ] || [ "${{ needs.ui_unit_tests.result }}" == "skipped" ]; then
echo "UI tests passed or skipped"
else
echo "UI tests failed"
exit 1
fi
echo "All tests completed successfully"