Files
greptimedb/.github/workflows/checks.yml
T
localhost 35ea88a4ef feat(ci): run query regression on ephemeral Aliyun ECS runners (#8937)
* feat(ci): add aliyun ecs ephemeral runner path for query regression

Signed-off-by: paomian <xpaomian@gmail.com>

* fix: improve condition for query-regression job execution in workflow

* feat: update Docker installation to use official repository and add GPG key handling

* Refactor query regression runner setup and configuration

- Removed deprecated PersistentVolumeClaim for build cache.
- Introduced a new bootstrap script for setting up the ECS runner host.
- Deleted obsolete Helm values files for runner configuration.
- Updated the Aliyun ECS runner provisioning script to reflect new cache paths.
- Modified GitHub workflows to use the new Aliyun ECS runner setup.
- Adjusted documentation to clarify the new runner lifecycle and provisioning process.

* fix: enhance runner service management during bootstrap process

* fix: update alibabacloud_tea_openapi dependency version in metadata

* feat: enhance ECS runner scripts with region_id and resource_group_id support

* fix: move containerd content store to data root for improved storage management

* feat: rename query-regression runner to ephemeral-github runner and update related scripts

* fix: update sentinel polling method to use serial console output for improved reliability

* fix: add environment variable checks for Alibaba Cloud access keys in ECS client

* fix: improve error handling in GitHub API requests for better diagnostics

* fix: improve cache disk detection logic for Aliyun ECS instances

* fix: enhance cache disk waiting logic with detailed output and error handling

* fix: update dependency version for alibabacloud_tea_openapi in teardown script

* fix: enhance cache disk waiting logic for better compatibility and clarity

* fix: enhance console output handling and add incremental logging during instance provisioning

* fix: add PATH environment variable for runner jobs in service and provision script

* fix: add machine telemetry sampling and logging during query regression jobs

* fix: update query regression documentation and provision script for cache disk handling

* fix: update SCCACHE_CACHE_SIZE validation to 10G for improved caching efficiency

* fix: remove outdated cache size checks and cleanup logic for fresh system disk runs

* fix: enhance instance deletion logic with region handling and console output export

* fix: add swap file setup and OOM handling for ECS runner to improve stability

* fix: update OOM handling and service restart logic for ECS runner to enhance stability

* fix: increase system disk size to 100 GiB for cold double nightly builds to prevent ENOSPC errors

* fix: increase system disk size to 150 GiB for ECS runner to prevent ENOSPC errors

* fix: add keep_instance option to preserve ECS instance for post-mortem debugging

* fix: disable unattended upgrades to prevent job cancellations during library updates

* fix: reduce system disk size to 40 GiB for ECS runner to prevent ENOSPC errors

* feat: Refactor Aliyun ECS runner provisioning and introduce nightly regression comparison

- Update `aliyun-ecs-runner-provision.py` to remove cache disk handling, simplifying the provisioning process.
- Introduce `query-regression-nightly-refs.py` to resolve and compare SHAs from successful nightly builds.
- Create `query-regression-nightly.yml` workflow to trigger nightly comparisons based on successful builds.
- Enhance `query-regression.yml` to include a `test-tooling` job for validating Python scripts before provisioning.
- Update tests for the new nightly reference selection logic and refactor existing tests to align with the new caching strategy.
- Modify documentation to reflect changes in caching and nightly comparison workflows.

* fix: enhance runner image tool verification with detailed checks

* fix: improve error handling in runner image tool verification

* fix: update tool versions in ECS image and workflow for consistency

* fix: correct typo in error message for unparseable ECS creation time

* fix: update README and workflow files for query regression tests and image hygiene

---------

Signed-off-by: paomian <xpaomian@gmail.com>
2026-08-26 12:11:14 +00:00

138 lines
4.3 KiB
YAML

on:
schedule:
- cron: "0 15 * * 1-5"
merge_group:
pull_request:
types: [ opened, synchronize, reopened, ready_for_review ]
paths-ignore:
- 'docs/**'
- 'config/**'
- '**.md'
- '.dockerignore'
- 'docker/**'
- '.gitignore'
- 'grafana/**'
- 'Makefile'
workflow_dispatch:
name: Checks
env:
TAPLO_VERSION: "0.9.3"
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true
jobs:
check-typos-and-docs:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' }}
name: Check typos and docs
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: crate-ci/typos@master
- name: Check the config docs
run: |
make config-docs && \
git diff --name-only --exit-code ./config/config.md \
|| (echo "'config/config.md' is not up-to-date, please run 'make config-docs'." && exit 1)
license-header-check:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' }}
runs-on: ubuntu-latest
name: Check License Header
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Check enterprise license lists
run: make check-enterprise-license
- uses: korandoru/hawkeye@v5
- name: Check enterprise license header
uses: korandoru/hawkeye@v5
with:
config: licenserc-enterprise.toml
github-script-tests:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' }}
name: GitHub Script Tests
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: actions/setup-node@v4
with:
node-version: 24
- name: Run Node GitHub script tests
run: node --test .github/scripts/query-regression-comment.test.cjs
- name: Setup uv
uses: astral-sh/setup-uv@v3
with:
version: "latest"
- name: Run check-version script tests
run: uv run --no-project python .github/scripts/check-version-test.py
- name: Run enterprise license check tests
run: python3 scripts/check-enterprise-license-test.py
- name: Run fuzz orchestration script tests
run: .github/scripts/run-fuzz-targets-test.sh
- name: Run query-regression Python tooling tests
run: |
python3 tests/perf/test_query_regression_runner_compaction_toctou.py
python3 tests/perf/test_query_regression_runner_otlp_trace_load.py
python3 tests/perf/test_query_regression_summary_otlp.py
python3 tests/perf/test_query_regression_case_selection.py
python3 tests/perf/test_query_regression_nightly_refs.py
python3 tests/perf/test_aliyun_ecs_runner_scripts.py
check:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' }}
name: Check
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ ubuntu-latest ]
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: arduino/setup-protoc@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- uses: actions-rust-lang/setup-rust-toolchain@v1
- name: Rust Cache
uses: Swatinem/rust-cache@v2
with:
# Shares across multiple jobs
# Shares with `Clippy` job
shared-key: "check-lint"
cache-all-crates: "true"
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Run cargo check
run: cargo check --locked --workspace --all-targets
- name: Run cargo check (all features)
run: cargo check --locked --workspace --all-targets --all-features
toml:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' }}
name: Toml Check
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: actions-rust-lang/setup-rust-toolchain@v1
- name: Install taplo
uses: taiki-e/install-action@v2
with:
tool: taplo@${{ env.TAPLO_VERSION }}
fallback: none
- name: Run taplo
run: taplo format --check