ci: reduce repeated Windows and fuzz tests on pull requests (#10043)

* ci: reduce repeated Windows and fuzz tests on pull requests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use the nightly merge label as the enqueue opt-in

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: log nightly merge decisions for dry-run verification

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: gate expensive jobs on GitHub script tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: grant pull request write permission for nightly label sync

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: run platform checks on schedule or manual dispatch

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(ci): update PR gate count for scheduled platform checks

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
This commit is contained in:
Weny Xu
2026-10-29 00:00:07 +00:00
committed by GitHub
parent efed847ca6
commit e87766d7ef
11 changed files with 290 additions and 126 deletions
@@ -0,0 +1,52 @@
name: GitHub Script Tests
description: Validate CI scripts before starting expensive jobs
runs:
using: composite
steps:
- uses: actions/setup-node@v4
with:
node-version: 24
- name: Run Node GitHub script tests
shell: bash
run: node --test .github/scripts/query-regression-comment.test.cjs .github/scripts/heavy-ci-policy.test.cjs .github/scripts/tests/nightly-merge.test.cjs
- name: Setup uv
uses: astral-sh/setup-uv@v3
with:
version: "latest"
- name: Run check-version script tests
shell: bash
run: uv run --no-project python .github/scripts/check-version-test.py
- name: Run release Linux architecture tests
shell: bash
run: python3 .github/scripts/release-linux-architectures-test.py
- name: Run previous-release-tag script tests
shell: bash
run: python3 .github/scripts/previous-release-tag-test.py
- name: Run enterprise license check tests
shell: bash
run: python3 scripts/check-enterprise-license-test.py
- name: Run fuzz orchestration script tests
shell: bash
run: .github/scripts/run-fuzz-targets-test.sh
- name: Run CI comment-command tests
shell: bash
run: python3 .github/scripts/tests/test_ci_slash.py
- name: Run CI scope gate tests
shell: bash
run: python3 .github/ci/tests/test_ci_scope.py
- name: Run metric snapshot benchmark tests
shell: bash
run: python3 -m unittest discover -s .github/scripts/tests -p 'test_metric_snapshot*.py'
- name: Run query-regression Python tooling tests
shell: bash
run: |
python3 tests/perf/test_query_regression_runner_compaction_toctou.py
python3 tests/perf/test_query_regression_runner_otlp_trace_load.py
python3 tests/perf/test_query_regression_summary_otlp.py
python3 tests/perf/test_query_regression_case_selection.py
python3 tests/perf/test_query_regression_nightly_refs.py
python3 tests/perf/test_query_regression_slash.py
python3 tests/perf/test_aliyun_ecs_runner_scripts.py
python3 tests/perf/test_ci_runner_scripts.py
python3 tests/perf/test_agent_observability_summary.py
python3 tests/perf/test_tracesbench_summary.py
+7 -5
View File
@@ -1,9 +1,10 @@
# Integration CI admission gate
`integration.yml` runs its expensive jobs — fuzz, export/import E2E, the sqlness
KvBackend legs — on every non-draft pull request. Most pull requests cannot
affect any of them. This directory decides which of those jobs start, using only
the diff.
`integration.yml` filters its expensive jobs — fuzz, export/import E2E, and the
sqlness KvBackend legs — by the diff. Fuzz additionally follows the event policy
in [Heavy CI scheduling](../heavy-ci.md): PR opening or ready-for-review events,
scheduled runs, and explicit manual profiles. A scope verdict cannot override
that event policy.
The gate can only *skip* work, never add it, and it fails open: when the verdict
is missing or uncertain, every gated job runs.
@@ -53,7 +54,8 @@ three minutes, less than the cost of getting its verdict wrong.
First match wins:
1. **Forced** — `merge_group`, `schedule`, `workflow_dispatch`, the `ci:full`
label or `CI_SCOPE_FORCE_ALL`. Everything runs.
label or `CI_SCOPE_FORCE_ALL`. All scope outputs allow execution; job event
conditions and manual profile selection still apply.
2. **Too large to judge** — more than 300 changed files. Everything runs.
3. **Always-run file changed** — `Cargo.toml`, `Cargo.lock`, `tests-fuzz/**`,
this directory, or anything else that changes what the suites build or how
+28
View File
@@ -0,0 +1,28 @@
# Heavy CI scheduling
Checks, Rust CI, and Integration CI each run the shared GitHub script test
action before expensive builds and tests. A failed, skipped, or cancelled
script-test job blocks those jobs, including manually dispatched runs. Each
workflow runs its own preflight so it does not wait on another workflow run.
Rust CI keeps the Windows compilation check on PRs and merge groups. Windows
nextest and Windows SQLness run in the existing Nightly CI workflow, which also
supports manual dispatch. Its existing schedule is Tuesday through Friday at
07:00 Asia/Shanghai, not every day. No new Windows test job is added.
Integration CI admits fuzz when a non-draft PR is opened or marked ready for
review, selecting suites with the existing diff-scope classifier. Its weekday
schedule (23:00 Asia/Shanghai, Monday through Friday) runs all fuzz suites. Ordinary pushes and reopening do not run fuzz; core
integration checks still run. Merge groups continue to skip fuzz.
Re-running an ordinary push-triggered Integration CI run does not enable fuzz.
A new push can cancel an earlier ready-for-review run. Use an explicit fuzz
profile through the CI entry points below when later changes need fuzz coverage.
For a same-repository branch, use **Run workflow** and select `fuzz_profile`
(`standalone`, `distributed`, `chaos`, or `all`). The default `none` runs core
integration without fuzz. The existing `/ci fuzz ...` commands remain available
for same-repository draft PRs with their existing authorization rules.
Validate job admission with `node --test .github/scripts/heavy-ci-policy.test.cjs`.
These tests verify triggering decisions, not runner availability or performance.
+4 -2
View File
@@ -2,14 +2,16 @@
The PR template opts new PRs into the `merge nightly` label. The Nightly Merge
workflow reconciles the label on PR creation, edits, and reopening. Existing PRs
without the checkbox are not opted in. Unchecking removes the label; it does not
without the checkbox can opt in by adding the label manually; label sync leaves
their labels unchanged. Unchecking removes the label; it does not
request an immediate merge or cancel an existing queue entry/auto-merge request.
Every day at 02:10, 03:10, 04:10, 05:10, 06:10, 07:10, and 08:10
Asia/Shanghai (UTC+8), the workflow enqueues up to five eligible PRs targeting
`main` per run, oldest first. Already queued PRs do not count toward this limit.
GitHub may delay scheduled runs.
PRs must still have the checkbox and label, be approved and clean, and have all
The label alone controls nightly opt-in; the enqueue job does not check the PR
body. PRs must be approved and clean, and have all
review threads resolved. Drafts are skipped.
Already queued PRs and manually enabled auto-merge requests are left alone;
Maintainers can use **Merge when ready** to proceed earlier. Unready PRs wait for a later
+105
View File
@@ -0,0 +1,105 @@
const { readFileSync } = require('node:fs');
const { resolve } = require('node:path');
const assert = require('node:assert/strict');
const { test } = require('node:test');
const workflows = resolve(__dirname, '../workflows');
const integration = readFileSync(resolve(workflows, 'integration.yml'), 'utf8');
const suites = ['fuzztest', 'unstable-fuzztest', 'distributed-fuzztest', 'distributed-fuzztest-with-chaos'];
const prerequisites = ['prepare-fuzz-targets', 'build-greptime-ci'];
function enabled(job, { event = 'pull_request', action = 'synchronize', draft = false, attempt = 1, profile = 'none', repository = 'GreptimeTeam/greptimedb', outputs = {}, results = {}, cancelled = false, workflow = integration } = {}) {
const block = workflow.split(` ${job}:\n`)[1].split(/\n [a-z][\w-]*:/)[0];
if (block.includes("needs['github-script-tests']")) {
assert.match(block, /^ needs: \[.*github-script-tests.*\]/m);
}
const expression = block.match(/^ if: \$\{\{ (.+) \}\}/m)[1];
// These job guards use only operators whose semantics match JS for this fixture.
return new Function('github', 'inputs', 'needs', 'cancelled', `return (${expression});`)(
{ repository, event_name: event, event: { action, pull_request: { draft } }, run_attempt: attempt },
{ fuzz_profile: profile, ci_command: false },
{ scope: { outputs }, ...Object.fromEntries(['github-script-tests', 'build', ...prerequisites].map(name => [name, { result: results[name] || 'success' }])) },
() => cancelled
);
}
test('fuzz event policy on ready PR admission and schedule, not every update', () => {
for (const job of [...suites, ...prerequisites]) {
for (const action of ['opened', 'ready_for_review']) assert.equal(enabled(job, { action }), true, job);
for (const action of ['synchronize', 'reopened']) assert.equal(enabled(job, { action }), false, job);
assert.equal(enabled(job, { event: 'schedule' }), true, job);
assert.equal(enabled(job, { event: 'merge_group' }), false, job);
assert.equal(enabled(job, { attempt: 2 }), false, job);
assert.equal(enabled(job, { action: 'opened', draft: true }), false, job);
assert.equal(enabled(job, { attempt: 2, draft: true }), false, job);
assert.equal(enabled(job, { event: 'schedule', repository: 'fork/greptimedb' }), false, job);
}
});
test('manual profiles select only their suite and all required build jobs', () => {
const expected = { none: [], standalone: suites.slice(0, 2), distributed: [suites[2]], chaos: [suites[3]], all: suites };
for (const [profile, jobs] of Object.entries(expected)) {
assert.deepEqual(suites.filter(job => enabled(job, { event: 'workflow_dispatch', profile })), jobs);
for (const job of prerequisites) assert.equal(enabled(job, { event: 'workflow_dispatch', profile }), profile !== 'none');
}
});
test('core integration tests still run on PR updates and merge groups', () => {
for (const job of ['build', 'sqlness', 'export-import-v2-e2e', 'run-multi-lang-tests']) {
assert.equal(enabled(job), true, job);
assert.equal(enabled(job, { event: 'merge_group' }), true, job);
}
});
test('Windows compilation stays in Rust CI; test coverage remains in Nightly CI', () => {
const rust = readFileSync(resolve(workflows, 'rust.yml'), 'utf8');
const nightly = readFileSync(resolve(workflows, 'nightly-ci.yml'), 'utf8');
assert.match(rust, /^ check-windows:/m);
assert.doesNotMatch(rust, /^ test-windows:/m);
assert.match(nightly, /^ test-on-windows:/m);
assert.match(nightly, /^ sqlness-windows:/m);
assert.match(nightly, /cargo nextest run -F dashboard/);
assert.match(nightly, /^ workflow_dispatch:/m);
});
test('scope exclusions and dependency failures compose with the event policy', () => {
const gates = {
'prepare-fuzz-targets': 'any_fuzz', 'build-greptime-ci': 'build_ci_needed',
fuzztest: 'fuzz_standalone', 'unstable-fuzztest': 'fuzz_unstable',
'distributed-fuzztest': 'fuzz_distributed', 'distributed-fuzztest-with-chaos': 'fuzz_chaos'
};
for (const [job, gate] of Object.entries(gates)) {
assert.equal(enabled(job, { action: 'opened', outputs: { [gate]: 'false' } }), false, job);
assert.equal(enabled(job, { action: 'opened', outputs: {} }), true, job);
assert.equal(enabled(job, { outputs: { [gate]: 'true' } }), false, job);
assert.equal(enabled(job, { action: 'opened', cancelled: true }), false, job);
}
for (const job of suites) {
for (const result of ['failure', 'skipped', 'cancelled']) {
assert.equal(enabled(job, { action: 'opened', results: { 'prepare-fuzz-targets': result } }), false, job);
const build = job === 'fuzztest' ? 'build' : 'build-greptime-ci';
assert.equal(enabled(job, { action: 'opened', results: { [build]: result } }), false, job);
}
}
});
test('script tests must pass before any expensive integration job can start', () => {
const jobs = ['build', 'sqlness', 'sqlness-kvbackend', 'export-import-v2-e2e',
'run-multi-lang-tests', 'compat', ...prerequisites, ...suites];
for (const job of jobs) {
assert.equal(enabled(job, { action: 'opened' }), true, job);
for (const result of ['failure', 'skipped', 'cancelled']) {
assert.equal(enabled(job, { action: 'opened', results: { 'github-script-tests': result } }), false, job);
}
}
});
test('script tests gate expensive Rust and Checks jobs', () => {
for (const [file, jobs] of [
['rust.yml', ['clippy', 'check-riscv64', 'check-windows', 'test', 'coverage']],
['checks.yml', ['check']]
]) {
const workflow = readFileSync(resolve(workflows, file), 'utf8');
for (const job of jobs) {
const event = job === 'coverage' ? 'merge_group' : 'workflow_dispatch';
assert.equal(enabled(job, { workflow, event }), true, job);
for (const result of ['failure', 'skipped', 'cancelled']) {
assert.equal(enabled(job, { workflow, event, results: { 'github-script-tests': result } }), false, job);
}
}
}
});
+20 -3
View File
@@ -61,16 +61,22 @@ async function run({ sync = false, body = checked, labels = ['merge nightly'],
const summary = { addHeading() { return this; }, addTable(rows) { calls.push(['summary', rows]); return this; }, async write() {} };
await scripts[sync ? 0 : 1](github,
{ repo: { owner: 'GreptimeTeam', repo: 'greptimedb' }, payload: { pull_request: { number: 1 } } },
{ summary, warning() {}, setFailed: message => calls.push(['failed', message]) },
{ summary, info() {}, warning() {}, setFailed: message => calls.push(['failed', message]) },
{ env: { DRY_RUN: String(dryRun) } });
return calls;
}
test('label sync grants PR label write permission without checking out PR code', () => {
const syncJob = workflow.split(' sync-label:\n')[1].split(' enqueue:\n')[0];
assert.match(syncJob, /^ pull-requests: write$/m);
assert.doesNotMatch(syncJob, /uses: actions\/checkout/);
});
test('template opts in and checkbox edits reconcile only the nightly label', async () => {
assert.ok(readFileSync(resolve(__dirname, '../../pull_request_template.md'), 'utf8').includes(checked));
assert.deepEqual(await run({ sync: true, labels: ['bug'] }), [['add', ['merge nightly']]]);
assert.deepEqual(await run({ sync: true }), []);
for (const body of [checked.replace('[x]', '[ ]'), '', null, `> ${checked}`]) {
for (const body of [checked.replace('[x]', '[ ]')]) {
assert.deepEqual(await run({ sync: true, body }), [['remove', 'merge nightly']]);
}
assert.deepEqual(await run({ sync: true, body: checked.replace('[x]', '[X]'), labels: [] }), [['add', ['merge nightly']]]);
@@ -83,7 +89,7 @@ test('eligible PR is enqueued with its checked head; dry run performs no mutatio
test('each eligibility gate independently prevents enqueueing', async () => {
for (const options of [
{ body: checked.replace('[x]', '[ ]') }, { labels: [] },
{ labels: [] },
{ current: { draft: true } }, { current: { state: 'closed' } }, { current: { base: { ref: 'release/v1.3' } } },
{ detail: { reviewDecision: 'REVIEW_REQUIRED' } }, { detail: { reviewDecision: 'CHANGES_REQUESTED' } },
{ detail: { mergeStateStatus: 'BLOCKED' } }, { detail: { mergeStateStatus: 'UNKNOWN' } },
@@ -106,3 +112,14 @@ test('per-run cap and individual failures do not prevent later candidates', asyn
assert.equal(calls.filter(c => c[0] === 'enqueue').length, 2);
assert.equal(calls.filter(c => c[0] === 'failed').length, 1);
});
test('label-only opt-in survives PRs without a template and does not require a checkbox', async () => {
for (const body of ['', null, 'Existing PR description', `> ${checked}`]) {
assert.deepEqual(await run({ sync: true, body }), []);
assert.deepEqual(await run({ sync: true, body, labels: [] }), []);
assert.equal((await run({ body })).filter(call => call[0] === 'enqueue').length, 1);
assert.equal((await run({ body, labels: [] })).some(call => call[0] === 'enqueue'), false);
}
// The label is authoritative for admission, even before checkbox synchronization.
assert.equal((await run({ body: checked.replace('[x]', '[ ]') })).some(call => call[0] === 'enqueue'), true);
});
+1 -1
View File
@@ -136,7 +136,7 @@ class CiSlashTest(unittest.TestCase):
def test_fork_rerun_gates_cover_standard_jobs(self):
import re
for name, count in [("rust", 7), ("integration", 7), ("checks", 6), ("docs", 3)]:
for name, count in [("rust", 6), ("integration", 8), ("checks", 6), ("docs", 3)]:
workflow = (SCRIPT.parents[1] / "workflows" / (name + ".yml")).read_text()
gates = re.findall(r"^ if:.*github.run_attempt > 1.*$", workflow, re.MULTILINE)
self.assertEqual(len(gates), count, name)
+3 -38
View File
@@ -71,43 +71,7 @@ jobs:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: actions/setup-node@v4
with:
node-version: 24
- name: Run Node GitHub script tests
run: node --test .github/scripts/query-regression-comment.test.cjs .github/scripts/tests/nightly-merge.test.cjs
- name: Setup uv
uses: astral-sh/setup-uv@v3
with:
version: "latest"
- name: Run check-version script tests
run: uv run --no-project python .github/scripts/check-version-test.py
- name: Run release Linux architecture tests
run: python3 .github/scripts/release-linux-architectures-test.py
- name: Run previous-release-tag script tests
run: python3 .github/scripts/previous-release-tag-test.py
- name: Run enterprise license check tests
run: python3 scripts/check-enterprise-license-test.py
- name: Run fuzz orchestration script tests
run: .github/scripts/run-fuzz-targets-test.sh
- name: Run CI comment-command tests
run: python3 .github/scripts/tests/test_ci_slash.py
- name: Run CI scope gate tests
run: python3 .github/ci/tests/test_ci_scope.py
- name: Run metric snapshot benchmark tests
run: python3 -m unittest discover -s .github/scripts/tests -p 'test_metric_snapshot*.py'
- name: Run query-regression Python tooling tests
run: |
python3 tests/perf/test_query_regression_runner_compaction_toctou.py
python3 tests/perf/test_query_regression_runner_otlp_trace_load.py
python3 tests/perf/test_query_regression_summary_otlp.py
python3 tests/perf/test_query_regression_case_selection.py
python3 tests/perf/test_query_regression_nightly_refs.py
python3 tests/perf/test_query_regression_slash.py
python3 tests/perf/test_aliyun_ecs_runner_scripts.py
python3 tests/perf/test_ci_runner_scripts.py
python3 tests/perf/test_agent_observability_summary.py
python3 tests/perf/test_tracesbench_summary.py
- uses: ./.github/actions/github-script-tests
scope-rules:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
@@ -127,7 +91,8 @@ jobs:
run: python3 .github/ci/ci_scope_rules.py --check-paths
check:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
if: ${{ needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
needs: [github-script-tests]
name: Check
runs-on: ${{ matrix.os }}
strategy:
+38 -24
View File
@@ -21,7 +21,7 @@ on:
default: false
type: boolean
fuzz_profile:
description: Fuzz profile selected by /ci fuzz
description: Fuzz profile for a manual run or /ci fuzz
required: false
default: none
type: choice
@@ -42,6 +42,17 @@ concurrency:
cancel-in-progress: true
jobs:
github-script-tests:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
name: GitHub Script Tests (Integration CI)
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: ./.github/actions/github-script-tests
# Which jobs this gate skips, and why: .github/ci/README.md.
scope:
name: CI Scope
@@ -94,7 +105,8 @@ jobs:
python3 .github/ci/ci_scope.py "${args[@]}"
build:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
if: ${{ needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
needs: [github-script-tests]
name: Build GreptimeDB binaries
runs-on: ${{ matrix.os }}
strategy:
@@ -155,9 +167,9 @@ jobs:
version: current
sqlness:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
if: ${{ needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
name: Sqlness Test (${{ matrix.mode.name }})
needs: build
needs: [github-script-tests, build]
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
@@ -213,9 +225,9 @@ jobs:
# required checks, so they are the only ones this gate may skip. They run
# the same cases as the `Basic` leg, so the diff only matters when it can
# reach the backend itself; the merge queue runs them regardless.
if: ${{ !cancelled() && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) && needs.build.result == 'success' && needs.scope.outputs.sqlness_kvbackend != 'false' }}
if: ${{ !cancelled() && needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) && needs.build.result == 'success' && needs.scope.outputs.sqlness_kvbackend != 'false' }}
name: Sqlness Test (${{ matrix.mode.name }})
needs: [scope, build]
needs: [github-script-tests, scope, build]
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
@@ -267,9 +279,9 @@ jobs:
retention-days: 3
export-import-v2-e2e:
if: ${{ !cancelled() && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) && needs.build.result == 'success' && needs.scope.outputs.export_import_v2 != 'false' }}
if: ${{ !cancelled() && needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) && needs.build.result == 'success' && needs.scope.outputs.export_import_v2 != 'false' }}
name: Export/Import V2 E2E Test
needs: [scope, build]
needs: [github-script-tests, scope, build]
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
@@ -337,9 +349,9 @@ jobs:
retention-days: 3
run-multi-lang-tests:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
if: ${{ needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
name: Run Multi-language SDK Tests
needs: build
needs: [github-script-tests, build]
uses: ./.github/workflows/run-multi-lang-tests.yml
with:
artifact-name: bins
@@ -362,9 +374,9 @@ jobs:
run: python3 .github/scripts/update-compat-versions.py --check --published-only --check-anchors
compat:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'merge_group' || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule' || (github.event_name == 'pull_request' && (github.event.action == 'ready_for_review' || (github.event.action == 'opened' && github.event.pull_request.draft == false)))) }}
if: ${{ needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'merge_group' || github.event_name == 'workflow_dispatch' || github.event_name == 'schedule' || (github.event_name == 'pull_request' && (github.event.action == 'ready_for_review' || (github.event.action == 'opened' && github.event.pull_request.draft == false)))) }}
name: Compatibility Test (recent releases)
needs: build
needs: [github-script-tests, build]
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -401,10 +413,12 @@ jobs:
path: /tmp/sqlness-compat*
retention-days: 3
# Fuzz runs on opening a ready PR, marking it ready, manual dispatch, and
# the scheduled main run. Synchronize events keep the core integration tests.
prepare-fuzz-targets:
if: ${{ !cancelled() && github.repository == 'GreptimeTeam/greptimedb' && ((github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name == 'workflow_dispatch' && inputs.ci_command) || github.event_name == 'merge_group') && (github.event_name == 'pull_request' || github.event_name == 'merge_group' || inputs.fuzz_profile != 'none') && needs.scope.outputs.any_fuzz != 'false' }}
if: ${{ !cancelled() && needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'schedule' || (github.event_name == 'pull_request' && github.event.pull_request.draft == false && (github.event.action == 'opened' || github.event.action == 'ready_for_review')) || (github.event_name == 'workflow_dispatch' && (inputs.fuzz_profile != 'none'))) && needs.scope.outputs.any_fuzz != 'false' }}
name: Prepare Fuzz Targets
needs: [scope]
needs: [github-script-tests, scope]
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
@@ -517,9 +531,9 @@ jobs:
if-no-files-found: error
fuzztest:
if: ${{ !cancelled() && github.repository == 'GreptimeTeam/greptimedb' && ((github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name == 'workflow_dispatch' && inputs.ci_command) || github.event_name == 'merge_group') && (github.event_name == 'pull_request' || github.event_name == 'merge_group' || inputs.fuzz_profile == 'standalone' || inputs.fuzz_profile == 'all') && needs.build.result == 'success' && needs['prepare-fuzz-targets'].result == 'success' && needs.scope.outputs.fuzz_standalone != 'false' }}
if: ${{ !cancelled() && needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'schedule' || (github.event_name == 'pull_request' && github.event.pull_request.draft == false && (github.event.action == 'opened' || github.event.action == 'ready_for_review')) || (github.event_name == 'workflow_dispatch' && (inputs.fuzz_profile == 'standalone' || inputs.fuzz_profile == 'all'))) && needs.build.result == 'success' && needs['prepare-fuzz-targets'].result == 'success' && needs.scope.outputs.fuzz_standalone != 'false' }}
name: Fuzz Test (Standalone, ${{ matrix.group.name }})
needs: [build, prepare-fuzz-targets, scope]
needs: [github-script-tests, build, prepare-fuzz-targets, scope]
runs-on: ubuntu-latest
timeout-minutes: 60
strategy:
@@ -598,9 +612,9 @@ jobs:
artifact-name: fuzz-standalone-${{ matrix.group.id }}
unstable-fuzztest:
if: ${{ !cancelled() && github.repository == 'GreptimeTeam/greptimedb' && ((github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name == 'workflow_dispatch' && inputs.ci_command) || github.event_name == 'merge_group') && (github.event_name == 'pull_request' || github.event_name == 'merge_group' || inputs.fuzz_profile == 'standalone' || inputs.fuzz_profile == 'all') && needs['build-greptime-ci'].result == 'success' && needs['prepare-fuzz-targets'].result == 'success' && needs.scope.outputs.fuzz_unstable != 'false' }}
if: ${{ !cancelled() && needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'schedule' || (github.event_name == 'pull_request' && github.event.pull_request.draft == false && (github.event.action == 'opened' || github.event.action == 'ready_for_review')) || (github.event_name == 'workflow_dispatch' && (inputs.fuzz_profile == 'standalone' || inputs.fuzz_profile == 'all'))) && needs['build-greptime-ci'].result == 'success' && needs['prepare-fuzz-targets'].result == 'success' && needs.scope.outputs.fuzz_unstable != 'false' }}
name: Unstable Fuzz Test
needs: [build-greptime-ci, prepare-fuzz-targets, scope]
needs: [github-script-tests, build-greptime-ci, prepare-fuzz-targets, scope]
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
@@ -658,9 +672,9 @@ jobs:
artifact-name: fuzz-unstable-standalone
build-greptime-ci:
if: ${{ !cancelled() && github.repository == 'GreptimeTeam/greptimedb' && ((github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name == 'workflow_dispatch' && inputs.ci_command && inputs.fuzz_profile != 'none') || github.event_name == 'merge_group') && needs.scope.outputs.build_ci_needed != 'false' }}
if: ${{ !cancelled() && needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'schedule' || (github.event_name == 'pull_request' && github.event.pull_request.draft == false && (github.event.action == 'opened' || github.event.action == 'ready_for_review')) || (github.event_name == 'workflow_dispatch' && (inputs.fuzz_profile != 'none'))) && needs.scope.outputs.build_ci_needed != 'false' }}
name: Build GreptimeDB binary (profile-CI)
needs: [scope]
needs: [github-script-tests, scope]
runs-on: ${{ matrix.os }}
strategy:
matrix:
@@ -719,10 +733,10 @@ jobs:
version: current
distributed-fuzztest:
if: ${{ !cancelled() && github.repository == 'GreptimeTeam/greptimedb' && ((github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name == 'workflow_dispatch' && inputs.ci_command) || github.event_name == 'merge_group') && (github.event_name == 'pull_request' || github.event_name == 'merge_group' || inputs.fuzz_profile == 'distributed' || inputs.fuzz_profile == 'all') && needs['build-greptime-ci'].result == 'success' && needs['prepare-fuzz-targets'].result == 'success' && needs.scope.outputs.fuzz_distributed != 'false' }}
if: ${{ !cancelled() && needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'schedule' || (github.event_name == 'pull_request' && github.event.pull_request.draft == false && (github.event.action == 'opened' || github.event.action == 'ready_for_review')) || (github.event_name == 'workflow_dispatch' && (inputs.fuzz_profile == 'distributed' || inputs.fuzz_profile == 'all'))) && needs['build-greptime-ci'].result == 'success' && needs['prepare-fuzz-targets'].result == 'success' && needs.scope.outputs.fuzz_distributed != 'false' }}
name: Fuzz Test (Distributed, ${{ matrix.mode.name }}, ${{ matrix.group.name }})
runs-on: ubuntu-latest
needs: [build-greptime-ci, prepare-fuzz-targets, scope]
needs: [github-script-tests, build-greptime-ci, prepare-fuzz-targets, scope]
timeout-minutes: 60
strategy:
fail-fast: false
@@ -958,10 +972,10 @@ jobs:
docker system prune -f
distributed-fuzztest-with-chaos:
if: ${{ !cancelled() && github.repository == 'GreptimeTeam/greptimedb' && ((github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name == 'workflow_dispatch' && inputs.ci_command) || github.event_name == 'merge_group') && (github.event_name == 'pull_request' || github.event_name == 'merge_group' || inputs.fuzz_profile == 'chaos' || inputs.fuzz_profile == 'all') && needs['build-greptime-ci'].result == 'success' && needs['prepare-fuzz-targets'].result == 'success' && needs.scope.outputs.fuzz_chaos != 'false' }}
if: ${{ !cancelled() && needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'schedule' || (github.event_name == 'pull_request' && github.event.pull_request.draft == false && (github.event.action == 'opened' || github.event.action == 'ready_for_review')) || (github.event_name == 'workflow_dispatch' && (inputs.fuzz_profile == 'chaos' || inputs.fuzz_profile == 'all'))) && needs['build-greptime-ci'].result == 'success' && needs['prepare-fuzz-targets'].result == 'success' && needs.scope.outputs.fuzz_chaos != 'false' }}
name: Fuzz Test with Chaos (Distributed, ${{ matrix.mode.name }}, ${{ matrix.group.name }})
runs-on: ubuntu-latest
needs: [build-greptime-ci, prepare-fuzz-targets, scope]
needs: [github-script-tests, build-greptime-ci, prepare-fuzz-targets, scope]
timeout-minutes: 60
strategy:
fail-fast: false
+9 -5
View File
@@ -23,7 +23,8 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
pull-requests: read
# PR label mutations require pull-request write permission.
pull-requests: write
issues: write
concurrency:
group: nightly-merge-label-${{ github.event.pull_request.number }}
@@ -36,7 +37,10 @@ jobs:
const { owner, repo } = context.repo;
const issue_number = context.payload.pull_request.number;
const { data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: issue_number });
const selected = /^- \[[xX]\] Merge during the nightly window <!-- nightly-merge -->\s*$/m.test(pr.body || '');
const checkbox = /^- \[([ xX])\] Merge during the nightly window <!-- nightly-merge -->\s*$/m.exec(pr.body || '');
// PRs without the template manage their opt-in through the label.
if (!checkbox) return;
const selected = checkbox[1].toLowerCase() === 'x';
const label = 'merge nightly';
const labeled = pr.labels.some(item => item.name === label);
if (selected && !labeled) {
@@ -73,11 +77,10 @@ jobs:
if (!candidate.pull_request || enqueued >= 5) continue;
const number = candidate.number;
try {
// Fetch current body/labels, not the earlier list snapshot.
// Fetch current labels and PR state, not the earlier list snapshot.
const { data: current } = await github.rest.pulls.get({ owner, repo, pull_number: number });
if (current.state !== 'open' || current.draft || current.base.ref !== 'main' ||
!current.labels.some(label => label.name === 'merge nightly') ||
!/^- \[[xX]\] Merge during the nightly window <!-- nightly-merge -->\s*$/m.test(current.body || '')) {
!current.labels.some(label => label.name === 'merge nightly')) {
rows.push([String(number), 'Skipped: not opted in, draft, closed, or not targeting main']);
continue;
}
@@ -134,5 +137,6 @@ jobs:
core.setFailed('Some PRs could not be processed; see workflow logs.');
}
}
for (const [number, result] of rows) core.info(`PR #${number}: ${result}`);
await core.summary.addHeading(dryRun ? 'Nightly merge (dry run)' : 'Nightly merge')
.addTable([['PR', 'Result'], ...rows]).write();
+23 -48
View File
@@ -31,6 +31,17 @@ concurrency:
cancel-in-progress: true
jobs:
github-script-tests:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
name: GitHub Script Tests (Rust CI)
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: ./.github/actions/github-script-tests
fmt:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
name: Rustfmt
@@ -50,7 +61,8 @@ jobs:
run: make fmt-check
clippy:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
if: ${{ needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
needs: [github-script-tests]
name: Clippy
runs-on: ubuntu-latest
timeout-minutes: 60
@@ -94,7 +106,9 @@ jobs:
run: make check-udeps
check-riscv64:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
# Keep platform coverage on the weekday schedule and allow manual checks.
if: ${{ needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') }}
needs: [github-script-tests]
name: Check riscv64 build
runs-on: ubuntu-latest
timeout-minutes: 90
@@ -120,7 +134,9 @@ jobs:
run: cargo check --locked --workspace --all-targets --features servers/dashboard --target riscv64gc-unknown-linux-gnu
check-windows:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
# Keep platform coverage on the weekday schedule and allow manual checks.
if: ${{ needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') }}
needs: [github-script-tests]
name: Check Windows build
runs-on: windows-2022
timeout-minutes: 60
@@ -148,48 +164,6 @@ jobs:
env:
RUSTFLAGS: "-D warnings"
test-windows:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false) }}
name: Test Windows
runs-on: windows-2022
timeout-minutes: 90
env:
CARGO_INCREMENTAL: 0
CARGO_PROFILE_DEV_DEBUG: 0
CARGO_PROFILE_TEST_DEBUG: 0
RUSTFLAGS: "-D warnings -C linker=lld-link"
RUST_BACKTRACE: 1
RUST_MIN_STACK: 8388608 # 8MB
steps:
- run: git config --global core.autocrlf false
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: ./.github/actions/setup-cyborg
- uses: taiki-e/install-action@v2
with:
tool: protoc@3.23.4
- uses: KyleMayes/install-llvm-action@v1
with:
version: "14.0"
- uses: actions-rust-lang/setup-rust-toolchain@v1
with:
cache: false
- name: Rust Cache
uses: Swatinem/rust-cache@v2
with:
shared-key: "test-windows"
cache-all-crates: "true"
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Install Cargo Nextest
uses: taiki-e/install-action@nextest
- name: Install WSL distribution
uses: Vampire/setup-wsl@v2
with:
distribution: Ubuntu-22.04
- name: Run Windows tests
run: cargo nextest run --locked --workspace -F dashboard --no-fail-fast
conflict-check:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
name: Check for conflict
@@ -202,10 +176,10 @@ jobs:
uses: olivernybroe/action-conflict-finder@v4.0
test:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && github.event_name != 'merge_group' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
if: ${{ needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && github.event_name != 'merge_group' && (github.event_name != 'pull_request' || github.event.pull_request.draft == false || github.run_attempt > 1) }}
runs-on: ubuntu-22.04-arm
timeout-minutes: 60
needs: [conflict-check, clippy, fmt, check-udeps]
needs: [github-script-tests, conflict-check, clippy, fmt, check-udeps]
steps:
- uses: actions/checkout@v4
with:
@@ -258,7 +232,8 @@ jobs:
UNITTEST_LOG_DIR: "__unittest_logs"
coverage:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && github.event_name == 'merge_group' }}
if: ${{ needs['github-script-tests'].result == 'success' && github.repository == 'GreptimeTeam/greptimedb' && github.event_name == 'merge_group' }}
needs: [github-script-tests]
runs-on: ubuntu-22.04-8-cores
timeout-minutes: 60
steps: