mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-11-06 19:06:58 +00:00
Weny Xu
ebb7e22566
* ci: gate the integration CI long tail by diff scope `integration.yml` starts its long tail on every pull request: the three fuzz families, the chaos legs, the export/import E2E job and the two sqlness KvBackend legs. Most diffs cannot affect them, so a `scope` job now classifies the pull request and tells those jobs whether to start. The classifier is deterministic and fail-open. Forced modes (merge queue, `schedule`, `workflow_dispatch`, `ci:full`, `CI_SCOPE_FORCE_ALL`) and three floors -- always-run paths, an uncovered `src/` change, and a PromQL/flow filter -- can only widen the result; everything else is a glob match against the suite catalog. Every step of the job is `continue-on-error` and every gate reads `needs.scope.outputs.<key> != 'false'`, so a classifier crash runs the jobs instead of skipping them. The rules are checked out from the pull request base, so a pull request cannot relax its own gate. The policy is data: `.github/ci/scope-rules.toml` holds the path groups, the suite catalog and the always-run list, with a `reason` on every exclusion. `ci_scope_rules.py` loads it with the stdlib `tomllib` and validates the shape (unknown or cyclic `@group` references, a `family` that disagrees with the suite id, an empty reason) and that every glob's literal prefix still exists, so a renamed crate cannot leave a rule that matches nothing. `make check-ci-scope` runs that check locally; the `CI Scope Rules` job runs it on `ubuntu-slim`, the same image the classifier uses. `.github/ci/scope-cases.toml` holds golden cases that `test_ci_scope.py` replays through the real classifier. The PostgreSQL and MySQL sqlness legs move into their own `sqlness-kvbackend` job so they can be gated: a job-level `if:` cannot read the matrix, and gating them inside `sqlness` would skip required checks and block the merge. They run the same cases as the required Basic leg, so only the backend itself matters. The verdict is visible where triage looks: one log line per suite, the exact gate values, `::warning::` when a lookup fails and the gate widens, and a job summary that lists each suite with the rule that decided it and the changed files that matched. Measured on 200 merged and open pull requests: 27.5% always run, 41% decided by path rules, 31.5% skip every gated job; 995 of 2600 suite selections are skipped, with no selection that a path rule makes. The classifier itself runs in about ten seconds on a single-vCPU runner. Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(ci): let the scope job read pull requests The `scope` job declared `permissions: contents: read`, and an explicit block sets every unlisted permission to `none`. `GET /pulls/{n}` therefore answered 403, `build_plan` widened to a full run as designed, and every gate read an absent output as "run": the gate could never skip anything. - `pull-requests: read` on the `scope` job. - `tests/runner/**` joins `@kv-backend`: the runner implements `--setup-pg`/`--setup-mysql`, which only the KV-backend legs pass, so a change there must not skip the only job that exercises it. The `UNMAPPED_CRATES` excuse is gone with it. - The README no longer claims a queue failure blocks the merge: the gated jobs are not required checks. Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(ci): run every suite when the kind registry script changes The distributed and chaos fuzz jobs start their cluster through `.github/actions/setup-kind`, which runs `.github/scripts/kind-with-registry.sh`. `.github/actions/**` is in `ALWAYS_RUN` but the script is not, so a change confined to it selected no suite at all and every gate stayed `false`: the jobs that exercise the changed cluster setup were skipped. The script joins `ALWAYS_RUN`, consistent with the action that calls it, and a golden case pins it. Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(ci): widen the gate for the fuzz corpus seeding script `fuzz-test/action.yaml` runs `.github/scripts/prepare-fuzz-corpus.sh` before the cache restore in every fuzz job, but the script was not in `ALWAYS_RUN`. A change confined to it therefore selected no suite: `any_fuzz` stayed `false` and every fuzz job that runs it was skipped. The script joins `ALWAYS_RUN` next to the other scripts the fuzz jobs execute, with a golden case for it. Both misses so far (this one and `kind-with-registry.sh`) came through a composite action, which the workflow never names. `WorkflowContractTests` now resolves `uses: ./.github/actions/<name>` transitively for the gated jobs and fails when one of the scripts they run is not in `ALWAYS_RUN`. Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com>