mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-08-18 12:08:22 +00:00
Lei, HUANG
7fa6f5f98e
* feat: add riscv64 cross-build support Add the missing build infrastructure for riscv64gc-unknown-linux-gnu. The codebase itself already compiles cleanly for riscv64 (verified with `cargo check --workspace --target riscv64gc-unknown-linux-gnu`): all architecture-sensitive dependencies support it (tikv-jemalloc-sys, aws-lc-sys, ring, pprof, simd-json). - .cargo/config.toml: set riscv64-linux-gnu-gcc as the linker for the riscv64gc-unknown-linux-gnu target - rust.yml: add a check-riscv64 CI job that cross-checks the whole workspace to prevent regressions from future dependency changes - docker/dev-builder/riscv64/Dockerfile: new cross dev-builder image with gcc/g++-riscv64-linux-gnu and the riscv64 rust target - Makefile: add dev-builder-riscv64 and build-riscv64-bin targets Verified end-to-end: the produced riscv64 binary starts in standalone mode under qemu and serves SQL (create/insert/select) over HTTP. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * ci: build riscv64 artifacts in the release workflow - release.yml: add build-linux-riscv64-artifacts job that cross-compiles greptime for riscv64gc-unknown-linux-gnu on the amd64 runner with the dev-builder-riscv64 image, and uploads greptime-linux-riscv64-* artifacts. The job is wired into the needs of publish-github-release, release-cn-artifacts and stop-linux-amd64-runner. Integration tests are skipped since the cross-compiled binary cannot run on the host. - release-dev-builder-images.yaml + build-dev-builder-images action: build and push the dev-builder-riscv64 image to DockerHub, and sync it to ECR and ACR via skopeo like the other dev-builder images. - Makefile: DEV_BUILDER_RISCV64_IMAGE_TAG now defaults to DEV_BUILDER_IMAGE_TAG so the existing tag-bump automation keeps the riscv64 image tag in sync. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: forward cargo extension in riscv64 build Pass CARGO_EXTENSION through build-riscv64-bin just like the existing build-by-dev-builder target, so wrappers such as sccache are preserved inside the cross-build container. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * ci: check riscv64 release feature graph Check all workspace targets with the servers/dashboard feature enabled so the riscv64 CI job covers the same optional dependency graph used by the release artifact build. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: gate riscv64 latest tags to main pushes Manual dev-builder workflow dispatches now publish only their immutable version tag. Update DockerHub and ECR latest tags only for the workflow's main-branch push event, preventing feature-branch builds from replacing the shared latest image. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * docs: include riscv64 in release input description Update the build_linux_artifacts workflow input description to reflect that it now triggers amd64, arm64, and riscv64 artifact builds. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: fall back when riscv64 builder is unpublished Before building a release artifact, pull the pinned RISC-V dev-builder from ECR. If the image has not been published yet, build the same tag locally from the current Dockerfile so releases remain unblocked during the builder-tag update window. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> --------- Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
825 lines
34 KiB
YAML
825 lines
34 KiB
YAML
name: Release
|
|
|
|
# There are two kinds of formal release:
|
|
# 1. The tag('v*.*.*') push release: the release workflow will be triggered by the tag push event.
|
|
# 2. The scheduled release(the version will be '${{ env.NEXT_RELEASE_VERSION }}-nightly-YYYYMMDD'): the release workflow will be triggered by the schedule event.
|
|
# A failed tag push release can be manually dispatched on the same tag to recover it without changing the formal release version.
|
|
on:
|
|
push:
|
|
tags:
|
|
- "v*.*.*"
|
|
schedule:
|
|
# At 00:00 on Monday.
|
|
- cron: '0 0 * * 1'
|
|
workflow_dispatch: # Allows you to run this workflow manually.
|
|
# Notes: The GitHub Actions ONLY support 10 inputs; this final input is the tenth.
|
|
inputs:
|
|
linux_amd64_runner:
|
|
type: choice
|
|
description: The runner uses to build linux-amd64 artifacts
|
|
default: ec2-c6i.4xlarge-amd64
|
|
options:
|
|
- ubuntu-22.04
|
|
- ubuntu-22.04-8-cores
|
|
- ubuntu-22.04-16-cores
|
|
- ubuntu-22.04-32-cores
|
|
- ubuntu-22.04-64-cores
|
|
- ec2-c6i.xlarge-amd64 # 4C8G
|
|
- ec2-c6i.2xlarge-amd64 # 8C16G
|
|
- ec2-c6i.4xlarge-amd64 # 16C32G
|
|
- ec2-c6i.8xlarge-amd64 # 32C64G
|
|
- ec2-c6i.16xlarge-amd64 # 64C128G
|
|
linux_arm64_runner:
|
|
type: choice
|
|
description: The runner uses to build linux-arm64 artifacts
|
|
default: ec2-c6g.8xlarge-arm64
|
|
options:
|
|
- ubuntu-2204-32-cores-arm
|
|
- ec2-c6g.xlarge-arm64 # 4C8G
|
|
- ec2-c6g.2xlarge-arm64 # 8C16G
|
|
- ec2-c6g.4xlarge-arm64 # 16C32G
|
|
- ec2-c6g.8xlarge-arm64 # 32C64G
|
|
- ec2-c6g.16xlarge-arm64 # 64C128G
|
|
macos_runner:
|
|
type: choice
|
|
description: The runner uses to build macOS artifacts
|
|
default: macos-latest
|
|
options:
|
|
- macos-latest
|
|
skip_test:
|
|
description: Do not run integration tests during the build
|
|
type: boolean
|
|
default: true
|
|
build_linux_artifacts:
|
|
type: boolean
|
|
description: Build linux artifacts (amd64, arm64, and riscv64)
|
|
required: false
|
|
default: false
|
|
build_macos_artifacts:
|
|
type: boolean
|
|
description: Build macos artifacts
|
|
required: false
|
|
default: false
|
|
build_windows_artifacts:
|
|
type: boolean
|
|
description: Build Windows artifacts
|
|
required: false
|
|
default: false
|
|
publish_github_release:
|
|
type: boolean
|
|
description: Create GitHub release and upload artifacts
|
|
required: false
|
|
default: false
|
|
release_images:
|
|
type: boolean
|
|
description: Build and push images to DockerHub and ACR
|
|
required: false
|
|
default: false
|
|
release_validation:
|
|
type: choice
|
|
description: Manual policy for accepted perf regressions or runner failures; automatic releases always run all validation
|
|
required: true
|
|
default: all
|
|
options:
|
|
- all
|
|
- skip-compat
|
|
- skip-query-regression
|
|
- skip-all
|
|
|
|
# Use env variables to control all the release process.
|
|
env:
|
|
# The arguments of building greptime.
|
|
CARGO_PROFILE: nightly
|
|
|
|
# Controls whether to run tests, include unit-test, integration-test and sqlness.
|
|
DISABLE_RUN_TESTS: ${{ inputs.skip_test || vars.DEFAULT_SKIP_TEST }}
|
|
|
|
# The scheduled version is '${{ env.NEXT_RELEASE_VERSION }}-nightly-YYYYMMDD', like v0.2.0-nightly-20230313;
|
|
NIGHTLY_RELEASE_PREFIX: nightly
|
|
|
|
jobs:
|
|
allocate-runners:
|
|
name: Allocate runners
|
|
if: ${{ github.repository == 'GreptimeTeam/greptimedb' }}
|
|
runs-on: ubuntu-latest
|
|
outputs:
|
|
linux-amd64-runner: ${{ steps.start-linux-amd64-runner.outputs.label }}
|
|
linux-arm64-runner: ${{ steps.start-linux-arm64-runner.outputs.label }}
|
|
macos-runner: ${{ inputs.macos_runner || vars.DEFAULT_MACOS_RUNNER }}
|
|
windows-runner: windows-2022-8-cores
|
|
|
|
# The following EC2 resource id will be used for resource releasing.
|
|
linux-amd64-ec2-runner-label: ${{ steps.start-linux-amd64-runner.outputs.label }}
|
|
linux-amd64-ec2-runner-instance-id: ${{ steps.start-linux-amd64-runner.outputs.ec2-instance-id }}
|
|
linux-arm64-ec2-runner-label: ${{ steps.start-linux-arm64-runner.outputs.label }}
|
|
linux-arm64-ec2-runner-instance-id: ${{ steps.start-linux-arm64-runner.outputs.ec2-instance-id }}
|
|
|
|
# The 'version' use as the global tag name of the release workflow.
|
|
version: ${{ steps.create-version.outputs.version }}
|
|
|
|
# The 'is-current-version-latest' determines whether to update 'latest' Docker tags and downstream repositories.
|
|
is-current-version-latest: ${{ steps.check-version.outputs.is-current-version-latest }}
|
|
is-current-version-stable: ${{ steps.check-version.outputs.is-current-version-stable }}
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- name: Check Rust toolchain version
|
|
shell: bash
|
|
run: |
|
|
./scripts/check-builder-rust-version.sh
|
|
|
|
# The create-version will create a global variable named 'version' in the global workflows.
|
|
# - If it's a tag push release or manual dispatch on a tag with REUSE_EXISTING_RELEASE_TAG=true, the version is the tag name(${{ github.ref_name }});
|
|
# - If it's a scheduled release, the version is '${{ env.NEXT_RELEASE_VERSION }}-nightly-$buildTime', like v0.2.0-nightly-20230313;
|
|
# - If it's a manual non-tag release, the version is '${{ env.NEXT_RELEASE_VERSION }}-<short-git-sha>-YYYYMMDDSS', like v0.2.0-e5b243c-2023071245;
|
|
- name: Create version
|
|
id: create-version
|
|
run: |
|
|
echo "version=$(./.github/scripts/create-version.sh)" >> $GITHUB_OUTPUT
|
|
env:
|
|
GITHUB_EVENT_NAME: ${{ github.event_name }}
|
|
GITHUB_REF_NAME: ${{ github.ref_name }}
|
|
GITHUB_REF_TYPE: ${{ github.ref_type }}
|
|
REUSE_EXISTING_RELEASE_TAG: "true"
|
|
NIGHTLY_RELEASE_PREFIX: ${{ env.NIGHTLY_RELEASE_PREFIX }}
|
|
|
|
- name: Check version
|
|
id: check-version
|
|
run: |
|
|
./.github/scripts/check-version.sh "${{ steps.create-version.outputs.version }}"
|
|
|
|
- name: Allocate linux-amd64 runner
|
|
if: ${{ inputs.build_linux_artifacts || inputs.publish_github_release || inputs.release_images || github.event_name == 'push' || github.event_name == 'schedule' }}
|
|
uses: ./.github/actions/start-runner
|
|
id: start-linux-amd64-runner
|
|
with:
|
|
runner: ${{ inputs.linux_amd64_runner || vars.DEFAULT_AMD64_RUNNER }}
|
|
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
|
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
|
aws-region: ${{ vars.EC2_RUNNER_REGION }}
|
|
github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
|
|
image-id: ${{ vars.EC2_RUNNER_LINUX_AMD64_IMAGE_ID }}
|
|
security-group-id: ${{ vars.EC2_RUNNER_SECURITY_GROUP_ID }}
|
|
subnet-id: ${{ vars.EC2_RUNNER_SUBNET_ID }}
|
|
|
|
- name: Allocate linux-arm64 runner
|
|
if: ${{ inputs.build_linux_artifacts || inputs.publish_github_release || inputs.release_images || github.event_name == 'push' || github.event_name == 'schedule' }}
|
|
uses: ./.github/actions/start-runner
|
|
id: start-linux-arm64-runner
|
|
with:
|
|
runner: ${{ inputs.linux_arm64_runner || vars.DEFAULT_ARM64_RUNNER }}
|
|
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
|
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
|
aws-region: ${{ vars.EC2_RUNNER_REGION }}
|
|
github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
|
|
image-id: ${{ vars.EC2_RUNNER_LINUX_ARM64_IMAGE_ID }}
|
|
security-group-id: ${{ vars.EC2_RUNNER_SECURITY_GROUP_ID }}
|
|
subnet-id: ${{ vars.EC2_RUNNER_SUBNET_ID }}
|
|
|
|
build-linux-amd64-artifacts:
|
|
name: Build linux-amd64 artifacts
|
|
if: ${{ inputs.build_linux_artifacts || inputs.publish_github_release || inputs.release_images || github.event_name == 'push' || github.event_name == 'schedule' }}
|
|
needs: [
|
|
allocate-runners,
|
|
]
|
|
runs-on: ${{ needs.allocate-runners.outputs.linux-amd64-runner }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- uses: ./.github/actions/build-linux-artifacts
|
|
with:
|
|
arch: amd64
|
|
cargo-profile: ${{ env.CARGO_PROFILE }}
|
|
version: ${{ needs.allocate-runners.outputs.version }}
|
|
disable-run-tests: ${{ env.DISABLE_RUN_TESTS }}
|
|
image-registry: ${{ vars.ECR_IMAGE_REGISTRY }}
|
|
image-namespace: ${{ vars.ECR_IMAGE_NAMESPACE }}
|
|
|
|
build-linux-arm64-artifacts:
|
|
name: Build linux-arm64 artifacts
|
|
if: ${{ inputs.build_linux_artifacts || inputs.publish_github_release || inputs.release_images || github.event_name == 'push' || github.event_name == 'schedule' }}
|
|
needs: [
|
|
allocate-runners,
|
|
]
|
|
runs-on: ${{ needs.allocate-runners.outputs.linux-arm64-runner }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- uses: ./.github/actions/build-linux-artifacts
|
|
with:
|
|
arch: arm64
|
|
cargo-profile: ${{ env.CARGO_PROFILE }}
|
|
version: ${{ needs.allocate-runners.outputs.version }}
|
|
disable-run-tests: ${{ env.DISABLE_RUN_TESTS }}
|
|
image-registry: ${{ vars.ECR_IMAGE_REGISTRY }}
|
|
image-namespace: ${{ vars.ECR_IMAGE_NAMESPACE }}
|
|
large-page-size: true
|
|
|
|
build-linux-riscv64-artifacts:
|
|
name: Build linux-riscv64 artifacts
|
|
if: ${{ inputs.build_linux_artifacts || inputs.publish_github_release || inputs.release_images || github.event_name == 'push' || github.event_name == 'schedule' }}
|
|
needs: [
|
|
allocate-runners,
|
|
]
|
|
runs-on: ${{ needs.allocate-runners.outputs.linux-amd64-runner }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- name: Prepare riscv64 dev-builder image
|
|
shell: bash
|
|
env:
|
|
IMAGE_REGISTRY: ${{ vars.ECR_IMAGE_REGISTRY }}
|
|
IMAGE_NAMESPACE: ${{ vars.ECR_IMAGE_NAMESPACE }}
|
|
run: |
|
|
builder_tag=$(grep '^DEV_BUILDER_IMAGE_TAG ?=' Makefile | awk '{print $3}')
|
|
builder_image="${IMAGE_REGISTRY}/${IMAGE_NAMESPACE}/dev-builder-riscv64:${builder_tag}"
|
|
if ! docker pull "${builder_image}"; then
|
|
make dev-builder-riscv64 \
|
|
IMAGE_REGISTRY="${IMAGE_REGISTRY}" \
|
|
IMAGE_NAMESPACE="${IMAGE_NAMESPACE}"
|
|
fi
|
|
|
|
# Cross-compile greptime for riscv64gc-unknown-linux-gnu on the amd64 runner
|
|
# with the riscv64 cross dev-builder image. Integration tests are skipped
|
|
# because the cross-compiled binary cannot run on the amd64 host.
|
|
- name: Build greptime riscv64 binary
|
|
shell: bash
|
|
run: |
|
|
make build-riscv64-bin \
|
|
CARGO_PROFILE=${{ env.CARGO_PROFILE }} \
|
|
FEATURES=servers/dashboard \
|
|
IMAGE_REGISTRY=${{ vars.ECR_IMAGE_REGISTRY }} \
|
|
IMAGE_NAMESPACE=${{ vars.ECR_IMAGE_NAMESPACE }}
|
|
|
|
- name: Upload artifacts
|
|
uses: ./.github/actions/upload-artifacts
|
|
with:
|
|
artifacts-dir: greptime-linux-riscv64-${{ needs.allocate-runners.outputs.version }}
|
|
target-files: ./target/riscv64gc-unknown-linux-gnu/${{ env.CARGO_PROFILE }}/greptime
|
|
version: ${{ needs.allocate-runners.outputs.version }}
|
|
|
|
build-macos-artifacts:
|
|
name: Build macOS artifacts
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- os: ${{ needs.allocate-runners.outputs.macos-runner }}
|
|
arch: aarch64-apple-darwin
|
|
features: servers/dashboard
|
|
artifacts-dir-prefix: greptime-darwin-arm64
|
|
- os: ${{ needs.allocate-runners.outputs.macos-runner }}
|
|
features: servers/dashboard
|
|
arch: x86_64-apple-darwin
|
|
artifacts-dir-prefix: greptime-darwin-amd64
|
|
runs-on: ${{ matrix.os }}
|
|
outputs:
|
|
build-macos-result: ${{ steps.set-build-macos-result.outputs.build-macos-result }}
|
|
needs: [
|
|
allocate-runners,
|
|
]
|
|
if: ${{ inputs.build_macos_artifacts || github.event_name == 'push' || github.event_name == 'schedule' }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- uses: ./.github/actions/build-macos-artifacts
|
|
with:
|
|
arch: ${{ matrix.arch }}
|
|
cargo-profile: ${{ env.CARGO_PROFILE }}
|
|
features: ${{ matrix.features }}
|
|
version: ${{ needs.allocate-runners.outputs.version }}
|
|
# We decide to disable the integration tests on macOS because it's unnecessary and time-consuming.
|
|
disable-run-tests: true
|
|
artifacts-dir: ${{ matrix.artifacts-dir-prefix }}-${{ needs.allocate-runners.outputs.version }}
|
|
|
|
- name: Set build macos result
|
|
id: set-build-macos-result
|
|
run: |
|
|
echo "build-macos-result=success" >> $GITHUB_OUTPUT
|
|
|
|
build-windows-artifacts:
|
|
name: Build Windows artifacts
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- os: ${{ needs.allocate-runners.outputs.windows-runner }}
|
|
arch: x86_64-pc-windows-msvc
|
|
features: servers/dashboard
|
|
artifacts-dir-prefix: greptime-windows-amd64
|
|
runs-on: ${{ matrix.os }}
|
|
outputs:
|
|
build-windows-result: ${{ steps.set-build-windows-result.outputs.build-windows-result }}
|
|
needs: [
|
|
allocate-runners,
|
|
]
|
|
if: ${{ inputs.build_windows_artifacts || github.event_name == 'push' || github.event_name == 'schedule' }}
|
|
steps:
|
|
- run: git config --global core.autocrlf false
|
|
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- uses: ./.github/actions/build-windows-artifacts
|
|
with:
|
|
arch: ${{ matrix.arch }}
|
|
cargo-profile: ${{ env.CARGO_PROFILE }}
|
|
features: ${{ matrix.features }}
|
|
version: ${{ needs.allocate-runners.outputs.version }}
|
|
disable-run-tests: ${{ env.DISABLE_RUN_TESTS }}
|
|
artifacts-dir: ${{ matrix.artifacts-dir-prefix }}-${{ needs.allocate-runners.outputs.version }}
|
|
|
|
- name: Set build windows result
|
|
id: set-build-windows-result
|
|
run: |
|
|
echo "build-windows-result=success" >> $Env:GITHUB_OUTPUT
|
|
|
|
prepare-release-validation:
|
|
name: Prepare release validation
|
|
if: ${{ inputs.publish_github_release || inputs.release_images || github.event_name == 'push' || github.event_name == 'schedule' }}
|
|
needs: [
|
|
allocate-runners,
|
|
]
|
|
runs-on: ubuntu-latest
|
|
outputs:
|
|
previous-release-tag: ${{ steps.refs.outputs.previous-release-tag }}
|
|
candidate-ref: ${{ steps.refs.outputs.candidate-ref }}
|
|
validation-policy: ${{ steps.refs.outputs.validation-policy }}
|
|
run-compat: ${{ steps.refs.outputs.run-compat }}
|
|
run-query-regression: ${{ steps.refs.outputs.run-query-regression }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- name: Resolve release validation refs
|
|
id: refs
|
|
shell: bash
|
|
env:
|
|
REQUESTED_VALIDATION_POLICY: ${{ inputs.release_validation }}
|
|
run: |
|
|
set -euo pipefail
|
|
base_commit="${GITHUB_SHA}"
|
|
if [[ "${GITHUB_REF_TYPE}" == "tag" ]]; then
|
|
base_commit="${GITHUB_SHA}^"
|
|
fi
|
|
previous_release_tag="$(git tag --merged "${base_commit}" --sort=-v:refname | grep -E '^v[0-9]+\.[0-9]+\.[0-9]+$' | head -n 1 || true)"
|
|
if [[ -z "${previous_release_tag}" ]]; then
|
|
previous_release_tag="main"
|
|
fi
|
|
|
|
if [[ "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ]]; then
|
|
validation_policy="${REQUESTED_VALIDATION_POLICY}"
|
|
else
|
|
validation_policy="all"
|
|
fi
|
|
case "${validation_policy}" in
|
|
all)
|
|
run_compat=true
|
|
run_query_regression=true
|
|
;;
|
|
skip-compat)
|
|
run_compat=false
|
|
run_query_regression=true
|
|
;;
|
|
skip-query-regression)
|
|
run_compat=true
|
|
run_query_regression=false
|
|
;;
|
|
skip-all)
|
|
run_compat=false
|
|
run_query_regression=false
|
|
;;
|
|
*)
|
|
printf 'Unsupported release validation policy: %s\n' "${validation_policy}" >&2
|
|
exit 1
|
|
;;
|
|
esac
|
|
|
|
echo "previous-release-tag=${previous_release_tag}" >> "${GITHUB_OUTPUT}"
|
|
echo "candidate-ref=${GITHUB_SHA}" >> "${GITHUB_OUTPUT}"
|
|
echo "validation-policy=${validation_policy}" >> "${GITHUB_OUTPUT}"
|
|
echo "run-compat=${run_compat}" >> "${GITHUB_OUTPUT}"
|
|
echo "run-query-regression=${run_query_regression}" >> "${GITHUB_OUTPUT}"
|
|
{
|
|
echo "## Release validation policy"
|
|
echo "- Effective policy: \`${validation_policy}\`"
|
|
echo "- Actor: \`${GITHUB_ACTOR}\`"
|
|
echo "- Triggering actor: \`${GITHUB_TRIGGERING_ACTOR}\`"
|
|
echo "- Event: \`${GITHUB_EVENT_NAME}\`"
|
|
echo "- Candidate SHA: \`${GITHUB_SHA}\`"
|
|
echo "- Baseline: \`${previous_release_tag}\`"
|
|
if [[ "${validation_policy}" != "all" ]]; then
|
|
echo "- ⚠️ Validation bypassed: compatibility=${run_compat}, query-regression=${run_query_regression}"
|
|
fi
|
|
} >> "${GITHUB_STEP_SUMMARY}"
|
|
|
|
compat-release:
|
|
name: Compatibility release test
|
|
if: ${{ needs.prepare-release-validation.outputs.run-compat == 'true' }}
|
|
needs: [
|
|
allocate-runners,
|
|
build-linux-amd64-artifacts,
|
|
prepare-release-validation,
|
|
]
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 120
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- uses: arduino/setup-protoc@v3
|
|
with:
|
|
repo-token: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
- uses: actions-rust-lang/setup-rust-toolchain@v1
|
|
with:
|
|
cache: false
|
|
|
|
- uses: Swatinem/rust-cache@v2
|
|
with:
|
|
shared-key: release-compat
|
|
cache-all-crates: "true"
|
|
save-if: false
|
|
|
|
- name: Download release greptime artifact
|
|
uses: actions/download-artifact@v4
|
|
with:
|
|
name: greptime-linux-amd64-${{ needs.allocate-runners.outputs.version }}
|
|
path: .
|
|
|
|
- name: Prepare release greptime binary
|
|
shell: bash
|
|
run: |
|
|
artifact_dir="greptime-linux-amd64-${{ needs.allocate-runners.outputs.version }}"
|
|
tar -xzf "${artifact_dir}.tar.gz"
|
|
mkdir -p ./bins
|
|
cp "${artifact_dir}/greptime" ./bins/greptime
|
|
chmod +x ./bins/greptime
|
|
./bins/greptime --version
|
|
|
|
- name: Build release compatibility runner
|
|
shell: bash
|
|
run: |
|
|
cargo build --profile "${CARGO_PROFILE}" -p sqlness-runner --bin sqlness-runner
|
|
cp "target/${CARGO_PROFILE}/sqlness-runner" ./bins/sqlness-runner
|
|
|
|
- name: Run release compatibility test
|
|
run: python3 .github/scripts/run-compat.py --preserve-state
|
|
|
|
- name: Upload compatibility failure logs
|
|
if: failure()
|
|
uses: actions/upload-artifact@v4
|
|
with:
|
|
name: release-compat-failure-logs
|
|
path: /tmp/sqlness-compat*
|
|
if-no-files-found: ignore
|
|
|
|
query-regression-release:
|
|
name: Query regression release test
|
|
if: ${{ needs.prepare-release-validation.outputs.run-query-regression == 'true' }}
|
|
needs: [
|
|
allocate-runners,
|
|
prepare-release-validation,
|
|
]
|
|
uses: ./.github/workflows/query-regression.yml
|
|
secrets: inherit
|
|
with:
|
|
case: all
|
|
base_ref: ${{ needs.prepare-release-validation.outputs.previous-release-tag }}
|
|
candidate_ref: ${{ needs.prepare-release-validation.outputs.candidate-ref }}
|
|
cargo_profile: nightly
|
|
runner: perf-regression-8-cores
|
|
|
|
release-images-to-dockerhub:
|
|
name: Build and push images to DockerHub
|
|
if: |
|
|
always() &&
|
|
(inputs.release_images || github.event_name == 'push' || github.event_name == 'schedule') &&
|
|
needs.allocate-runners.result == 'success' &&
|
|
needs.build-linux-amd64-artifacts.result == 'success' &&
|
|
needs.build-linux-arm64-artifacts.result == 'success' &&
|
|
needs.prepare-release-validation.result == 'success' &&
|
|
(needs.compat-release.result == 'success' ||
|
|
(needs.prepare-release-validation.outputs.run-compat == 'false' && needs.compat-release.result == 'skipped')) &&
|
|
(needs.query-regression-release.result == 'success' ||
|
|
(needs.prepare-release-validation.outputs.run-query-regression == 'false' && needs.query-regression-release.result == 'skipped'))
|
|
needs: [
|
|
allocate-runners,
|
|
build-linux-amd64-artifacts,
|
|
build-linux-arm64-artifacts,
|
|
prepare-release-validation,
|
|
compat-release,
|
|
query-regression-release,
|
|
]
|
|
runs-on: ubuntu-latest
|
|
outputs:
|
|
build-image-result: ${{ steps.set-build-image-result.outputs.build-image-result }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- name: Build and push images to dockerhub
|
|
uses: ./.github/actions/build-images
|
|
with:
|
|
image-registry: docker.io
|
|
image-namespace: ${{ vars.IMAGE_NAMESPACE }}
|
|
image-name: ${{ vars.GREPTIMEDB_IMAGE_NAME }}
|
|
image-registry-username: ${{ secrets.DOCKERHUB_USERNAME }}
|
|
image-registry-password: ${{ secrets.DOCKERHUB_TOKEN }}
|
|
version: ${{ needs.allocate-runners.outputs.version }}
|
|
push-latest-tag: ${{ needs.allocate-runners.outputs.is-current-version-latest == 'true' && needs.allocate-runners.outputs.is-current-version-stable == 'true' && github.ref_type == 'tag' && !contains(github.ref_name, 'nightly') && github.event_name != 'schedule' }}
|
|
|
|
- name: Set build image result
|
|
id: set-build-image-result
|
|
run: |
|
|
echo "build-image-result=success" >> $GITHUB_OUTPUT
|
|
|
|
release-cn-artifacts:
|
|
name: Release artifacts to CN region
|
|
if: ${{ inputs.release_images || github.event_name == 'push' || github.event_name == 'schedule' }}
|
|
needs: [ # The job have to wait for all the artifacts are built.
|
|
allocate-runners,
|
|
build-linux-amd64-artifacts,
|
|
build-linux-arm64-artifacts,
|
|
build-macos-artifacts,
|
|
build-linux-riscv64-artifacts,
|
|
build-windows-artifacts,
|
|
release-images-to-dockerhub,
|
|
]
|
|
runs-on: ubuntu-latest-16-cores
|
|
# When we push to ACR, it's easy to fail due to some unknown network issues.
|
|
# However, we don't want to fail the whole workflow because of this.
|
|
# The ACR have daily sync with DockerHub, so don't worry about the image not being updated.
|
|
continue-on-error: true
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- name: Release artifacts to CN region
|
|
uses: ./.github/actions/release-cn-artifacts
|
|
with:
|
|
src-image-registry: docker.io
|
|
src-image-namespace: ${{ vars.IMAGE_NAMESPACE }}
|
|
src-image-name: ${{ vars.GREPTIMEDB_IMAGE_NAME }}
|
|
dst-image-registry-username: ${{ secrets.ALICLOUD_USERNAME }}
|
|
dst-image-registry-password: ${{ secrets.ALICLOUD_PASSWORD }}
|
|
dst-image-registry: ${{ vars.ACR_IMAGE_REGISTRY }}
|
|
dst-image-namespace: ${{ vars.IMAGE_NAMESPACE }}
|
|
version: ${{ needs.allocate-runners.outputs.version }}
|
|
proxy-url: ${{ secrets.PROXY_URL }}
|
|
proxy-username: ${{ secrets.PROXY_USERNAME }}
|
|
proxy-password: ${{ secrets.PROXY_PASSWORD }}
|
|
dev-mode: false
|
|
upload-to-s3: true
|
|
update-version-info: true
|
|
push-latest-tag: ${{ needs.allocate-runners.outputs.is-current-version-latest == 'true' && needs.allocate-runners.outputs.is-current-version-stable == 'true' && github.ref_type == 'tag' && !contains(github.ref_name, 'nightly') && github.event_name != 'schedule' }}
|
|
|
|
publish-github-release:
|
|
name: Create GitHub release and upload artifacts
|
|
# Use always() to run even when optional jobs (macos, windows) are skipped.
|
|
# Then check that required jobs succeeded and optional jobs didn't fail.
|
|
if: |
|
|
always() &&
|
|
(inputs.publish_github_release || github.event_name == 'push' || github.event_name == 'schedule') &&
|
|
needs.allocate-runners.result == 'success' &&
|
|
needs.prepare-release-validation.result == 'success' &&
|
|
(needs.compat-release.result == 'success' ||
|
|
(needs.prepare-release-validation.outputs.run-compat == 'false' && needs.compat-release.result == 'skipped')) &&
|
|
(needs.query-regression-release.result == 'success' ||
|
|
(needs.prepare-release-validation.outputs.run-query-regression == 'false' && needs.query-regression-release.result == 'skipped')) &&
|
|
needs.build-linux-amd64-artifacts.result == 'success' &&
|
|
needs.build-linux-arm64-artifacts.result == 'success' &&
|
|
needs.build-linux-riscv64-artifacts.result == 'success' &&
|
|
(needs.build-macos-artifacts.result == 'success' || needs.build-macos-artifacts.result == 'skipped') &&
|
|
(needs.build-windows-artifacts.result == 'success' || needs.build-windows-artifacts.result == 'skipped') &&
|
|
(needs.release-images-to-dockerhub.result == 'success' ||
|
|
(github.event_name == 'workflow_dispatch' && inputs.release_images == false &&
|
|
needs.release-images-to-dockerhub.result == 'skipped')) &&
|
|
(github.event_name != 'workflow_dispatch' || github.ref_type != 'tag' ||
|
|
needs.allocate-runners.outputs.is-current-version-stable != 'true' ||
|
|
needs.allocate-runners.outputs.is-current-version-latest == 'true')
|
|
needs: [ # The job have to wait for all the artifacts are built.
|
|
allocate-runners,
|
|
build-linux-amd64-artifacts,
|
|
build-linux-arm64-artifacts,
|
|
build-macos-artifacts,
|
|
build-linux-riscv64-artifacts,
|
|
build-windows-artifacts,
|
|
release-images-to-dockerhub,
|
|
prepare-release-validation,
|
|
compat-release,
|
|
query-regression-release,
|
|
]
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- name: Publish GitHub release
|
|
uses: ./.github/actions/publish-github-release
|
|
with:
|
|
version: ${{ needs.allocate-runners.outputs.version }}
|
|
|
|
update-compat-versions:
|
|
name: Update compatibility versions
|
|
needs: [allocate-runners, publish-github-release]
|
|
if: ${{ needs.publish-github-release.result == 'success' }}
|
|
uses: ./.github/workflows/update-compat-versions.yml
|
|
with:
|
|
release_tag: ${{ needs.allocate-runners.outputs.version }}
|
|
secrets: inherit
|
|
|
|
### Stop runners ###
|
|
# It's very necessary to split the job of releasing runners into 'stop-linux-amd64-runner' and 'stop-linux-arm64-runner'.
|
|
# Because we can terminate the specified EC2 instance immediately after the job is finished without unnecessary waiting.
|
|
stop-linux-amd64-runner: # It's always run as the last job in the workflow to make sure that the runner is released.
|
|
name: Stop linux-amd64 runner
|
|
# Only run this job when the runner is allocated.
|
|
if: ${{ always() }}
|
|
runs-on: ubuntu-latest
|
|
needs: [
|
|
allocate-runners,
|
|
build-linux-amd64-artifacts,
|
|
build-linux-riscv64-artifacts,
|
|
]
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- name: Stop EC2 runner
|
|
uses: ./.github/actions/stop-runner
|
|
with:
|
|
label: ${{ needs.allocate-runners.outputs.linux-amd64-ec2-runner-label }}
|
|
ec2-instance-id: ${{ needs.allocate-runners.outputs.linux-amd64-ec2-runner-instance-id }}
|
|
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
|
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
|
aws-region: ${{ vars.EC2_RUNNER_REGION }}
|
|
github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
|
|
|
|
stop-linux-arm64-runner: # It's always run as the last job in the workflow to make sure that the runner is released.
|
|
name: Stop linux-arm64 runner
|
|
# Only run this job when the runner is allocated.
|
|
if: ${{ always() }}
|
|
runs-on: ubuntu-latest
|
|
needs: [
|
|
allocate-runners,
|
|
build-linux-arm64-artifacts,
|
|
]
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
|
|
- name: Stop EC2 runner
|
|
uses: ./.github/actions/stop-runner
|
|
with:
|
|
label: ${{ needs.allocate-runners.outputs.linux-arm64-ec2-runner-label }}
|
|
ec2-instance-id: ${{ needs.allocate-runners.outputs.linux-arm64-ec2-runner-instance-id }}
|
|
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
|
|
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
|
|
aws-region: ${{ vars.EC2_RUNNER_REGION }}
|
|
github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
|
|
|
|
bump-downstream-repo-versions:
|
|
name: Bump downstream repo versions
|
|
if: ${{ github.event_name == 'schedule' || ((github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref_type == 'tag' && needs.allocate-runners.outputs.is-current-version-stable == 'true') }}
|
|
needs: [allocate-runners, publish-github-release]
|
|
runs-on: ubuntu-latest
|
|
# Permission reference: https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
|
|
permissions:
|
|
issues: write # Allows the action to create issues for cyborg.
|
|
contents: write # Allows the action to create a release.
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
- uses: ./.github/actions/setup-cyborg
|
|
- name: Bump downstream repo versions
|
|
working-directory: cyborg
|
|
run: pnpm tsx bin/bump-versions.ts
|
|
env:
|
|
TARGET_REPOS: website,docs,demo
|
|
VERSION: ${{ needs.allocate-runners.outputs.version }}
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
WEBSITE_REPO_TOKEN: ${{ secrets.WEBSITE_REPO_TOKEN }}
|
|
DOCS_REPO_TOKEN: ${{ secrets.DOCS_REPO_TOKEN }}
|
|
DEMO_REPO_TOKEN: ${{ secrets.DEMO_REPO_TOKEN }}
|
|
|
|
bump-helm-charts-version:
|
|
name: Bump helm charts version
|
|
if: ${{ github.ref_type == 'tag' && !contains(github.ref_name, 'nightly') && github.event_name != 'schedule' && needs.allocate-runners.outputs.is-current-version-stable == 'true' && needs.allocate-runners.outputs.is-current-version-latest == 'true' }}
|
|
needs: [allocate-runners, publish-github-release]
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: write
|
|
pull-requests: write
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
|
|
- name: Bump helm charts version
|
|
env:
|
|
GITHUB_TOKEN: ${{ secrets.HELM_CHARTS_REPO_TOKEN }}
|
|
VERSION: ${{ needs.allocate-runners.outputs.version }}
|
|
run: |
|
|
./.github/scripts/update-helm-charts-version.sh
|
|
|
|
bump-homebrew-greptime-version:
|
|
name: Bump homebrew greptime version
|
|
if: ${{ github.ref_type == 'tag' && !contains(github.ref_name, 'nightly') && github.event_name != 'schedule' && needs.allocate-runners.outputs.is-current-version-stable == 'true' && needs.allocate-runners.outputs.is-current-version-latest == 'true' }}
|
|
needs: [allocate-runners, publish-github-release]
|
|
runs-on: ubuntu-latest
|
|
permissions:
|
|
contents: write
|
|
pull-requests: write
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
|
|
- name: Bump homebrew greptime version
|
|
env:
|
|
GITHUB_TOKEN: ${{ secrets.HOMEBREW_GREPTIME_REPO_TOKEN }}
|
|
VERSION: ${{ needs.allocate-runners.outputs.version }}
|
|
run: |
|
|
./.github/scripts/update-homebrew-greptme-version.sh
|
|
|
|
notification:
|
|
if: ${{ github.repository == 'GreptimeTeam/greptimedb' && (github.event_name == 'push' || github.event_name == 'schedule') && always() }}
|
|
name: Send notification to Greptime team
|
|
needs: [
|
|
release-images-to-dockerhub,
|
|
build-macos-artifacts,
|
|
build-windows-artifacts,
|
|
]
|
|
runs-on: ubuntu-latest
|
|
# Permission reference: https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
|
|
permissions:
|
|
issues: write # Allows the action to create issues for cyborg.
|
|
contents: write # Allows the action to create a release.
|
|
env:
|
|
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL_DEVELOP_CHANNEL }}
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
persist-credentials: false
|
|
- uses: ./.github/actions/setup-cyborg
|
|
- name: Report CI status
|
|
id: report-ci-status
|
|
working-directory: cyborg
|
|
run: pnpm tsx bin/report-ci-failure.ts
|
|
env:
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
CI_REPORT_STATUS: ${{ needs.release-images-to-dockerhub.outputs.build-image-result == 'success' && needs.build-windows-artifacts.outputs.build-windows-result == 'success' && needs.build-macos-artifacts.outputs.build-macos-result == 'success' }}
|
|
- name: Notify release successful result
|
|
uses: slackapi/slack-github-action@v1.25.0
|
|
if: ${{ needs.release-images-to-dockerhub.outputs.build-image-result == 'success' && needs.build-windows-artifacts.outputs.build-windows-result == 'success' && needs.build-macos-artifacts.outputs.build-macos-result == 'success' }}
|
|
with:
|
|
payload: |
|
|
{"text": "GreptimeDB's release version has completed successfully."}
|
|
|
|
- name: Notify release failed result
|
|
uses: slackapi/slack-github-action@v1.25.0
|
|
if: ${{ needs.release-images-to-dockerhub.outputs.build-image-result != 'success' || needs.build-windows-artifacts.outputs.build-windows-result != 'success' || needs.build-macos-artifacts.outputs.build-macos-result != 'success' }}
|
|
with:
|
|
payload: |
|
|
{"text": "GreptimeDB's release version has failed, please check ${{ steps.report-ci-status.outputs.html_url }}."}
|