mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-10-03 10:35:35 +00:00
revert(ci): pin the query-regression runner toolchain to nightly-2026-03-21 (#9389)
* revert(ci): pin the query-regression runner toolchain to nightly-2026-03-21 The query-regression benchmark compiles both the candidate and the BASE checkout (the previous nightly build). Base refs can predate the stable-toolchain migration and still use #![feature] gates, so the runner toolchain must stay a nightly that can build historical revisions; deriving it from the workspace rust-toolchain.toml (now stable 1.96.1) breaks base builds. Revert the toolchain-toml coupling introduced in #9369 and keep the parts that were correct: - query-regression.yml: RUSTUP_TOOLCHAIN hard-pinned to nightly-2026-03-21 again (with a comment explaining why), Verify assertions back to the exact nightly versions, and the test-tooling pin-derivation machinery removed - runner Dockerfile: ARG RUST_TOOLCHAIN=nightly-2026-03-21 + baked ENV restored; the COPY rust-toolchain.toml parsing removed - build-ecs-image.py: the toml staging in the builder user-data removed; base-image auto-resolution kept but retargeted to Ubuntu 26.04 to match the Verify tool pins (python3 3.14 etc.) - the rebuild job keeps the epoch-bump lockstep, now as a PR from a timestamped ci/ branch mirroring update-dev-builder-version.sh (direct pushes to main fail GH006 under branch protection) Complements #9376 (actions-runner bump), which requires a rebuilt image with the non-deprecated runner. Part of #9289. Signed-off-by: Ning Sun <sunning@greptime.com> * chore: skip qreg for rust-toolchain change --------- Signed-off-by: Ning Sun <sunning@greptime.com>
This commit is contained in:
Ning Sun
@@ -120,23 +120,12 @@ jobs:
|
||||
# Ordinary PRs also run the same tests from checks.yml.
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
outputs:
|
||||
# The Rust toolchain pin, resolved from rust-toolchain.toml so the
|
||||
# benchmark always runs the workspace toolchain without a second
|
||||
# hard-coded copy in this workflow.
|
||||
rust_toolchain: ${{ steps.rust-toolchain.outputs.pin }}
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Resolve Rust toolchain pin
|
||||
id: rust-toolchain
|
||||
run: |
|
||||
pin="$(grep -E '^channel' rust-toolchain.toml | cut -d'"' -f2)"
|
||||
echo "pin=${pin}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Test query regression tooling
|
||||
run: |
|
||||
python3 tests/perf/test_query_regression_runner_compaction_toctou.py
|
||||
@@ -201,7 +190,13 @@ jobs:
|
||||
CARGO_HOME: /home/runner/.cargo
|
||||
UV_CACHE_DIR: /home/runner/.cargo/uv-cache
|
||||
RUSTUP_HOME: /opt/rustup
|
||||
RUSTUP_TOOLCHAIN: ${{ needs.test-tooling.outputs.rust_toolchain }}
|
||||
# Deliberately NOT derived from rust-toolchain.toml: the benchmark
|
||||
# also compiles the BASE checkout (the previous nightly build, which
|
||||
# may predate the stable-toolchain migration and still uses
|
||||
# #![feature] gates), so the runner must keep a nightly toolchain
|
||||
# that can build historical revisions. This pin moves only via a
|
||||
# deliberate runner-image rebuild.
|
||||
RUSTUP_TOOLCHAIN: nightly-2026-03-21
|
||||
RUSTUP_AUTO_INSTALL: "0"
|
||||
CARGO_TARGET_DIR: /home/runner/query-regression-target
|
||||
QUERY_REGRESSION_CACHE_META: /home/runner/query-regression-cache-meta
|
||||
@@ -515,16 +510,17 @@ jobs:
|
||||
require_eq cargo_path "$(command -v cargo || true)" "/opt/cargo/bin/cargo"
|
||||
require_eq rustc_path "$(command -v rustc || true)" "/opt/cargo/bin/rustc"
|
||||
require_match rustup "$(capture rustup --version)" '^rustup[[:space:]]1\.29\.0([[:space:]]|$)'
|
||||
# The toolchain pin flows from rust-toolchain.toml (resolved in the
|
||||
# test-tooling job); escape it for the regex assertions below.
|
||||
pin_regex="${RUSTUP_TOOLCHAIN//./\\.}"
|
||||
# The runner toolchain is pinned to nightly-2026-03-21 (see the
|
||||
# RUSTUP_TOOLCHAIN comment): it must compile historical base
|
||||
# checkouts, so it intentionally does not follow rust-toolchain.toml.
|
||||
require_match cargo "$(capture cargo --version)" \
|
||||
"^cargo[[:space:]]${pin_regex}[[:space:]]\([0-9a-f]+[[:space:]][0-9]{4}-[0-9]{2}-[0-9]{2}\)$"
|
||||
'^cargo[[:space:]]1\.96\.0-nightly[[:space:]]\(cbb9bb8bd[[:space:]][0-9]{4}-[0-9]{2}-[0-9]{2}\)$'
|
||||
require_match rustc "$(capture rustc --version)" \
|
||||
"^rustc[[:space:]]${pin_regex}[[:space:]]\([0-9a-f]+[[:space:]][0-9]{4}-[0-9]{2}-[0-9]{2}\)$"
|
||||
'^rustc[[:space:]]1\.96\.0-nightly[[:space:]]\(ac7f9ec7d[[:space:]][0-9]{4}-[0-9]{2}-[0-9]{2}\)$'
|
||||
require_match active_toolchain "$(capture rustup show active-toolchain)" \
|
||||
"^${pin_regex}-x86_64-unknown-linux-gnu([[:space:]]|$)"
|
||||
'^nightly-2026-03-21-x86_64-unknown-linux-gnu([[:space:]]|$)'
|
||||
require_eq RUSTUP_HOME "${RUSTUP_HOME}" "/opt/rustup"
|
||||
require_eq RUSTUP_TOOLCHAIN "${RUSTUP_TOOLCHAIN}" "nightly-2026-03-21"
|
||||
require_eq RUSTUP_AUTO_INSTALL "${RUSTUP_AUTO_INSTALL}" "0"
|
||||
require "readable /opt/rustup" test -r /opt/rustup
|
||||
require "executable /opt/rustup" test -x /opt/rustup
|
||||
|
||||
@@ -71,9 +71,13 @@ jobs:
|
||||
files="$(git diff --name-only "${base}" "${head}")"
|
||||
dev_builder=false
|
||||
query_regression_runner=false
|
||||
# rust-toolchain.toml only gates the dev-builder images: they bake
|
||||
# the workspace toolchain. The query-regression runner toolchain is
|
||||
# pinned inside its Dockerfile (nightly-2026-03-21) and must NOT
|
||||
# follow the workspace pin, so its rebuild triggers only on changes
|
||||
# under its own directory.
|
||||
if grep -qx 'rust-toolchain.toml' <<<"${files}"; then
|
||||
dev_builder=true
|
||||
query_regression_runner=true
|
||||
fi
|
||||
if grep -q '^docker/dev-builder/' <<<"${files}"; then
|
||||
dev_builder=true
|
||||
@@ -335,18 +339,21 @@ jobs:
|
||||
# Rebuilds the query-regression ECS runner image via the ops tool in
|
||||
# .github/runner-scale-sets/query-regression/ecs-image/: boots a
|
||||
# temporary pay-as-you-go ECS builder, snapshots a new custom image,
|
||||
# then completes the documented lockstep updates in order:
|
||||
# then completes the documented lockstep updates:
|
||||
# 1. bumps RUNNER_IMAGE_EPOCH in query-regression.yml (target-cache
|
||||
# invalidation) and pushes the commit to main, and only then
|
||||
# invalidation) via a PR from a timestamped ci/ branch (main is
|
||||
# branch-protected; mirrors update-dev-builder-version.sh), and
|
||||
# 2. points the repo variable QUERY_REGRESSION_ECS_IMAGE_ID at the
|
||||
# new image, so the next regression run picks up image and epoch
|
||||
# together.
|
||||
# new image. The runner toolchain is pinned inside the Dockerfile
|
||||
# (nightly-2026-03-21, independent of rust-toolchain.toml — the
|
||||
# benchmark must compile historical base checkouts), so image
|
||||
# rebuilds do not change the cache ABI.
|
||||
#
|
||||
# Required repository configuration (same names the provisioning job
|
||||
# uses): vars ALIYUN_ECS_REGION_ID, ALIYUN_ECS_VSWITCH_ID,
|
||||
# ALIYUN_ECS_SECURITY_GROUP_ID, optionally ALIYUN_ECS_RESOURCE_GROUP_ID
|
||||
# and ALIYUN_ECS_BASE_IMAGE_ID (deterministic base-image pin; otherwise
|
||||
# the rebuild auto-resolves the latest public Ubuntu 24.04 image).
|
||||
# the rebuild auto-resolves the latest public Ubuntu 26.04 image).
|
||||
# Secrets: ALICLOUD_ECS_ACCESS_KEY_ID, ALICLOUD_ECS_ACCESS_KEY_SECRET,
|
||||
# GH_PERSONAL_ACCESS_TOKEN (repo push + actions-variable write).
|
||||
name: Rebuild query-regression runner image
|
||||
@@ -392,7 +399,7 @@ jobs:
|
||||
echo "## Rebuilt runner image" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "New image: \`${image_id}\`" >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Bump RUNNER_IMAGE_EPOCH and update the image id variable
|
||||
- name: Open the epoch-bump PR and update the image id variable
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
|
||||
run: |
|
||||
@@ -405,21 +412,42 @@ jobs:
|
||||
exit 1
|
||||
fi
|
||||
new_epoch=$((old_epoch + 1))
|
||||
|
||||
# main is branch-protected (required reviews + status checks), so
|
||||
# the epoch bump cannot be pushed there directly. Mirror
|
||||
# .github/scripts/update-dev-builder-version.sh: timestamped bot
|
||||
# branch, plain push, and a PR with reviewers. Do NOT use [skip ci]
|
||||
# on the commit: the required checks must be able to run for the PR
|
||||
# to become mergeable.
|
||||
BRANCH="ci/update-query-regression-epoch-$(date +%Y%m%d%H%M%S)"
|
||||
git config user.name "greptimedb-ci"
|
||||
git config user.email "greptimedb-ci@greptime.com"
|
||||
git fetch origin main
|
||||
git checkout -b "${BRANCH}" origin/main
|
||||
sed -i "s/readonly RUNNER_IMAGE_EPOCH=\"${old_epoch}\"/readonly RUNNER_IMAGE_EPOCH=\"${new_epoch}\"/" \
|
||||
.github/workflows/query-regression.yml
|
||||
|
||||
git config user.name "greptimedb-ci"
|
||||
git config user.email "greptimedb-ci@users.noreply.github.com"
|
||||
git add .github/workflows/query-regression.yml
|
||||
git commit -m "ci(query-regression): bump RUNNER_IMAGE_EPOCH to ${new_epoch} for image ${image_id} [skip ci]"
|
||||
git pull --rebase origin main
|
||||
git push origin HEAD:main
|
||||
git commit -s -m "ci(query-regression): bump RUNNER_IMAGE_EPOCH to ${new_epoch} for image ${image_id}"
|
||||
git push origin "${BRANCH}"
|
||||
|
||||
# Only point the variable at the new image after the epoch commit
|
||||
# landed, so the next run picks up image and epoch together.
|
||||
gh pr create \
|
||||
--title "ci(query-regression): bump RUNNER_IMAGE_EPOCH to ${new_epoch}" \
|
||||
--body "Auto-generated by the \`Rebuild query-regression runner image\` job in ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID} for image \`${image_id}\`. Merging this invalidates the query-regression target cache (RUNNER_IMAGE_EPOCH ${old_epoch} → ${new_epoch}). The \`QUERY_REGRESSION_ECS_IMAGE_ID\` repo variable already points at the new image; the epoch bump is belt-and-suspenders for image-content drift." \
|
||||
--base main \
|
||||
--head "${BRANCH}" \
|
||||
--reviewer sunng87 \
|
||||
--reviewer daviderli614 \
|
||||
--reviewer killme2008 \
|
||||
--reviewer evenyag \
|
||||
--reviewer fengjiachun \
|
||||
--reviewer WenyXu
|
||||
|
||||
# Point the variable at the new image once the PR exists. Image
|
||||
# rebuilds keep the same pinned toolchain, so the cache ABI is
|
||||
# unchanged and not waiting for the PR merge is fine.
|
||||
gh api -X PATCH "repos/${{ github.repository }}/actions/variables/QUERY_REGRESSION_ECS_IMAGE_ID" \
|
||||
-f value="${image_id}"
|
||||
|
||||
echo "## Lockstep updates" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "- QUERY_REGRESSION_ECS_IMAGE_ID → \`${image_id}\`" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "- RUNNER_IMAGE_EPOCH → \`${new_epoch}\` (committed to main)" >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "- RUNNER_IMAGE_EPOCH → \`${new_epoch}\` (PR on \`${BRANCH}\`, pending review/merge)" >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
Reference in New Issue
Block a user