feat: add riscv64 cross-build support (#8820)

* feat: add riscv64 cross-build support

Add the missing build infrastructure for riscv64gc-unknown-linux-gnu.
The codebase itself already compiles cleanly for riscv64 (verified with
`cargo check --workspace --target riscv64gc-unknown-linux-gnu`): all
architecture-sensitive dependencies support it (tikv-jemalloc-sys,
aws-lc-sys, ring, pprof, simd-json).

- .cargo/config.toml: set riscv64-linux-gnu-gcc as the linker for the
  riscv64gc-unknown-linux-gnu target
- rust.yml: add a check-riscv64 CI job that cross-checks the whole
  workspace to prevent regressions from future dependency changes
- docker/dev-builder/riscv64/Dockerfile: new cross dev-builder image
  with gcc/g++-riscv64-linux-gnu and the riscv64 rust target
- Makefile: add dev-builder-riscv64 and build-riscv64-bin targets

Verified end-to-end: the produced riscv64 binary starts in standalone
mode under qemu and serves SQL (create/insert/select) over HTTP.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* ci: build riscv64 artifacts in the release workflow

- release.yml: add build-linux-riscv64-artifacts job that cross-compiles
  greptime for riscv64gc-unknown-linux-gnu on the amd64 runner with the
  dev-builder-riscv64 image, and uploads greptime-linux-riscv64-*
  artifacts. The job is wired into the needs of publish-github-release,
  release-cn-artifacts and stop-linux-amd64-runner. Integration tests
  are skipped since the cross-compiled binary cannot run on the host.
- release-dev-builder-images.yaml + build-dev-builder-images action:
  build and push the dev-builder-riscv64 image to DockerHub, and sync
  it to ECR and ACR via skopeo like the other dev-builder images.
- Makefile: DEV_BUILDER_RISCV64_IMAGE_TAG now defaults to
  DEV_BUILDER_IMAGE_TAG so the existing tag-bump automation keeps the
  riscv64 image tag in sync.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix: forward cargo extension in riscv64 build

Pass CARGO_EXTENSION through build-riscv64-bin just like the existing
build-by-dev-builder target, so wrappers such as sccache are preserved
inside the cross-build container.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* ci: check riscv64 release feature graph

Check all workspace targets with the servers/dashboard feature enabled so
the riscv64 CI job covers the same optional dependency graph used by the
release artifact build.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix: gate riscv64 latest tags to main pushes

Manual dev-builder workflow dispatches now publish only their immutable
version tag. Update DockerHub and ECR latest tags only for the workflow's
main-branch push event, preventing feature-branch builds from replacing
the shared latest image.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* docs: include riscv64 in release input description

Update the build_linux_artifacts workflow input description to reflect
that it now triggers amd64, arm64, and riscv64 artifact builds.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix: fall back when riscv64 builder is unpublished

Before building a release artifact, pull the pinned RISC-V dev-builder
from ECR. If the image has not been published yet, build the same tag
locally from the current Dockerfile so releases remain unblocked during
the builder-tag update window.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

---------

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
This commit is contained in:
Lei, HUANG
2026-08-10 16:11:21 +08:00
committed by GitHub
parent d4af650ec0
commit 7fa6f5f98e
7 changed files with 229 additions and 2 deletions
+3
View File
@@ -1,6 +1,9 @@
[target.aarch64-unknown-linux-gnu]
linker = "aarch64-linux-gnu-gcc"
[target.riscv64gc-unknown-linux-gnu]
linker = "riscv64-linux-gnu-gcc"
[alias]
sqlness = "run --bin sqlness-runner --target-dir target/sqlness --"
@@ -31,6 +31,10 @@ inputs:
description: Build dev-builder-android image
required: false
default: "true"
build-dev-builder-riscv64:
description: Build dev-builder-riscv64 image
required: false
default: "true"
runs:
using: composite
steps:
@@ -70,6 +74,21 @@ runs:
IMAGE_NAMESPACE=${{ inputs.dockerhub-image-namespace }} \
DEV_BUILDER_IMAGE_TAG=${{ inputs.version }}
- name: Build and push dev-builder-riscv64 image # Only build image for amd64 platform (it cross-compiles riscv64 binaries).
shell: bash
if: ${{ inputs.build-dev-builder-riscv64 == 'true' }}
run: |
make dev-builder-riscv64 \
IMAGE_REGISTRY=${{ inputs.dockerhub-image-registry }} \
IMAGE_NAMESPACE=${{ inputs.dockerhub-image-namespace }} \
DEV_BUILDER_RISCV64_IMAGE_TAG=${{ inputs.version }}
docker push ${{ inputs.dockerhub-image-registry }}/${{ inputs.dockerhub-image-namespace }}/dev-builder-riscv64:${{ inputs.version }}
if [[ "${{ github.event_name }}" == "push" ]]; then
docker tag ${{ inputs.dockerhub-image-registry }}/${{ inputs.dockerhub-image-namespace }}/dev-builder-riscv64:${{ inputs.version }} \
${{ inputs.dockerhub-image-registry }}/${{ inputs.dockerhub-image-namespace }}/dev-builder-riscv64:latest
docker push ${{ inputs.dockerhub-image-registry }}/${{ inputs.dockerhub-image-namespace }}/dev-builder-riscv64:latest
fi
- name: Build and push dev-builder-android image # Only build image for amd64 platform.
shell: bash
if: ${{ inputs.build-dev-builder-android == 'true' }}
@@ -24,6 +24,11 @@ on:
description: Release dev-builder-android image
required: false
default: false
release_dev_builder_riscv64_image:
type: boolean
description: Release dev-builder-riscv64 image
required: false
default: false
update_dev_builder_image_tag:
type: boolean
description: Update the DEV_BUILDER_IMAGE_TAG in Makefile and create a PR
@@ -36,7 +41,7 @@ jobs:
# The jobs are triggered by the following events:
# 1. Manually triggered workflow_dispatch event
# 2. Push event when the PR that modifies the `rust-toolchain.toml` or `docker/dev-builder/**` is merged to main
if: ${{ github.event_name == 'push' || inputs.release_dev_builder_ubuntu_image || inputs.release_dev_builder_centos_image || inputs.release_dev_builder_android_image }}
if: ${{ github.event_name == 'push' || inputs.release_dev_builder_ubuntu_image || inputs.release_dev_builder_centos_image || inputs.release_dev_builder_android_image || inputs.release_dev_builder_riscv64_image }}
runs-on: ubuntu-latest
outputs:
version: ${{ steps.set-version.outputs.version }}
@@ -68,6 +73,7 @@ jobs:
build-dev-builder-ubuntu: ${{ inputs.release_dev_builder_ubuntu_image || github.event_name == 'push' }}
build-dev-builder-centos: ${{ inputs.release_dev_builder_centos_image || github.event_name == 'push' }}
build-dev-builder-android: ${{ inputs.release_dev_builder_android_image || github.event_name == 'push' }}
build-dev-builder-riscv64: ${{ inputs.release_dev_builder_riscv64_image || github.event_name == 'push' }}
release-dev-builder-images-ecr:
name: Release dev builder images to AWS ECR
@@ -154,6 +160,35 @@ jobs:
copy -a docker://docker.io/$IMAGE_NAMESPACE/dev-builder-android:latest \
docker://$ECR_IMAGE_REGISTRY/$ECR_IMAGE_NAMESPACE/dev-builder-android:latest
- name: Push dev-builder-riscv64 image
shell: bash
if: ${{ inputs.release_dev_builder_riscv64_image || github.event_name == 'push' }}
env:
IMAGE_VERSION: ${{ needs.release-dev-builder-images.outputs.version }}
IMAGE_NAMESPACE: ${{ vars.IMAGE_NAMESPACE }}
ECR_IMAGE_REGISTRY: ${{ vars.ECR_IMAGE_REGISTRY }}
ECR_IMAGE_NAMESPACE: ${{ vars.ECR_IMAGE_NAMESPACE }}
run: |
docker run -v "${DOCKER_CONFIG:-$HOME/.docker}:/root/.docker:ro" \
-e "REGISTRY_AUTH_FILE=/root/.docker/config.json" \
quay.io/skopeo/stable:latest \
copy -a docker://docker.io/$IMAGE_NAMESPACE/dev-builder-riscv64:$IMAGE_VERSION \
docker://$ECR_IMAGE_REGISTRY/$ECR_IMAGE_NAMESPACE/dev-builder-riscv64:$IMAGE_VERSION
- name: Push latest dev-builder-riscv64 image
shell: bash
if: ${{ github.event_name == 'push' }}
env:
IMAGE_NAMESPACE: ${{ vars.IMAGE_NAMESPACE }}
ECR_IMAGE_REGISTRY: ${{ vars.ECR_IMAGE_REGISTRY }}
ECR_IMAGE_NAMESPACE: ${{ vars.ECR_IMAGE_NAMESPACE }}
run: |
docker run -v "${DOCKER_CONFIG:-$HOME/.docker}:/root/.docker:ro" \
-e "REGISTRY_AUTH_FILE=/root/.docker/config.json" \
quay.io/skopeo/stable:latest \
copy -a docker://docker.io/$IMAGE_NAMESPACE/dev-builder-riscv64:latest \
docker://$ECR_IMAGE_REGISTRY/$ECR_IMAGE_NAMESPACE/dev-builder-riscv64:latest
release-dev-builder-images-cn: # Note: Be careful issue: https://github.com/containers/skopeo/issues/1874 and we decide to use the latest stable skopeo container.
name: Release dev builder images to CN region
runs-on: ubuntu-latest
@@ -209,6 +244,19 @@ jobs:
quay.io/skopeo/stable:latest \
copy -a docker://docker.io/$IMAGE_NAMESPACE/dev-builder-android:$IMAGE_VERSION \
docker://$ACR_IMAGE_REGISTRY/$IMAGE_NAMESPACE/dev-builder-android:$IMAGE_VERSION
- name: Push dev-builder-riscv64 image
shell: bash
if: ${{ inputs.release_dev_builder_riscv64_image || github.event_name == 'push' }}
env:
IMAGE_VERSION: ${{ needs.release-dev-builder-images.outputs.version }}
IMAGE_NAMESPACE: ${{ vars.IMAGE_NAMESPACE }}
ACR_IMAGE_REGISTRY: ${{ vars.ACR_IMAGE_REGISTRY }}
run: |
docker run -v "${DOCKER_CONFIG:-$HOME/.docker}:/root/.docker:ro" \
-e "REGISTRY_AUTH_FILE=/root/.docker/config.json" \
quay.io/skopeo/stable:latest \
copy -a docker://docker.io/$IMAGE_NAMESPACE/dev-builder-riscv64:$IMAGE_VERSION \
docker://$ACR_IMAGE_REGISTRY/$IMAGE_NAMESPACE/dev-builder-riscv64:$IMAGE_VERSION
update-dev-builder-image-tag:
name: Update dev-builder image tag
+51 -1
View File
@@ -52,7 +52,7 @@ on:
default: true
build_linux_artifacts:
type: boolean
description: Build linux artifacts (both amd64 and arm64)
description: Build linux artifacts (amd64, arm64, and riscv64)
required: false
default: false
build_macos_artifacts:
@@ -225,6 +225,52 @@ jobs:
image-namespace: ${{ vars.ECR_IMAGE_NAMESPACE }}
large-page-size: true
build-linux-riscv64-artifacts:
name: Build linux-riscv64 artifacts
if: ${{ inputs.build_linux_artifacts || inputs.publish_github_release || inputs.release_images || github.event_name == 'push' || github.event_name == 'schedule' }}
needs: [
allocate-runners,
]
runs-on: ${{ needs.allocate-runners.outputs.linux-amd64-runner }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
persist-credentials: false
- name: Prepare riscv64 dev-builder image
shell: bash
env:
IMAGE_REGISTRY: ${{ vars.ECR_IMAGE_REGISTRY }}
IMAGE_NAMESPACE: ${{ vars.ECR_IMAGE_NAMESPACE }}
run: |
builder_tag=$(grep '^DEV_BUILDER_IMAGE_TAG ?=' Makefile | awk '{print $3}')
builder_image="${IMAGE_REGISTRY}/${IMAGE_NAMESPACE}/dev-builder-riscv64:${builder_tag}"
if ! docker pull "${builder_image}"; then
make dev-builder-riscv64 \
IMAGE_REGISTRY="${IMAGE_REGISTRY}" \
IMAGE_NAMESPACE="${IMAGE_NAMESPACE}"
fi
# Cross-compile greptime for riscv64gc-unknown-linux-gnu on the amd64 runner
# with the riscv64 cross dev-builder image. Integration tests are skipped
# because the cross-compiled binary cannot run on the amd64 host.
- name: Build greptime riscv64 binary
shell: bash
run: |
make build-riscv64-bin \
CARGO_PROFILE=${{ env.CARGO_PROFILE }} \
FEATURES=servers/dashboard \
IMAGE_REGISTRY=${{ vars.ECR_IMAGE_REGISTRY }} \
IMAGE_NAMESPACE=${{ vars.ECR_IMAGE_NAMESPACE }}
- name: Upload artifacts
uses: ./.github/actions/upload-artifacts
with:
artifacts-dir: greptime-linux-riscv64-${{ needs.allocate-runners.outputs.version }}
target-files: ./target/riscv64gc-unknown-linux-gnu/${{ env.CARGO_PROFILE }}/greptime
version: ${{ needs.allocate-runners.outputs.version }}
build-macos-artifacts:
name: Build macOS artifacts
strategy:
@@ -520,6 +566,7 @@ jobs:
build-linux-amd64-artifacts,
build-linux-arm64-artifacts,
build-macos-artifacts,
build-linux-riscv64-artifacts,
build-windows-artifacts,
release-images-to-dockerhub,
]
@@ -568,6 +615,7 @@ jobs:
(needs.prepare-release-validation.outputs.run-query-regression == 'false' && needs.query-regression-release.result == 'skipped')) &&
needs.build-linux-amd64-artifacts.result == 'success' &&
needs.build-linux-arm64-artifacts.result == 'success' &&
needs.build-linux-riscv64-artifacts.result == 'success' &&
(needs.build-macos-artifacts.result == 'success' || needs.build-macos-artifacts.result == 'skipped') &&
(needs.build-windows-artifacts.result == 'success' || needs.build-windows-artifacts.result == 'skipped') &&
(needs.release-images-to-dockerhub.result == 'success' ||
@@ -581,6 +629,7 @@ jobs:
build-linux-amd64-artifacts,
build-linux-arm64-artifacts,
build-macos-artifacts,
build-linux-riscv64-artifacts,
build-windows-artifacts,
release-images-to-dockerhub,
prepare-release-validation,
@@ -619,6 +668,7 @@ jobs:
needs: [
allocate-runners,
build-linux-amd64-artifacts,
build-linux-riscv64-artifacts,
]
steps:
- name: Checkout
+26
View File
@@ -96,6 +96,32 @@ jobs:
- name: Check unused dependencies
run: make check-udeps
check-riscv64:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' }}
name: Check riscv64 build
runs-on: ubuntu-latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: arduino/setup-protoc@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Install riscv64 cross toolchain
run: sudo apt-get update && sudo apt-get install -y gcc-riscv64-linux-gnu g++-riscv64-linux-gnu
- uses: actions-rust-lang/setup-rust-toolchain@v1
- name: Add riscv64 target
run: rustup target add riscv64gc-unknown-linux-gnu
- name: Rust Cache
uses: Swatinem/rust-cache@v2
with:
shared-key: "check-riscv64"
cache-all-crates: "true"
save-if: ${{ github.ref == 'refs/heads/main' }}
- name: Run cargo check for riscv64
run: cargo check --locked --workspace --all-targets --features servers/dashboard --target riscv64gc-unknown-linux-gnu
conflict-check:
if: ${{ github.repository == 'GreptimeTeam/greptimedb' }}
name: Check for conflict
+26
View File
@@ -9,6 +9,9 @@ IMAGE_REGISTRY ?= docker.io
IMAGE_NAMESPACE ?= greptime
IMAGE_TAG ?= latest
DEV_BUILDER_IMAGE_TAG ?= 2026-03-21-9c9d9e9e-20260331090344
DEV_BUILDER_RISCV64_IMAGE_TAG ?= $(DEV_BUILDER_IMAGE_TAG)
DEV_BUILDER_RISCV64_BASE_IMAGE ?= ubuntu:22.04
RISCV64_TARGET ?= riscv64gc-unknown-linux-gnu
BUILDX_MULTI_PLATFORM_BUILD ?= false
BUILDX_BUILDER_NAME ?= gtbuilder
BASE_IMAGE ?= ubuntu
@@ -98,6 +101,21 @@ build-by-dev-builder: ## Build greptime by dev-builder.
RELEASE=${RELEASE} \
CARGO_BUILD_EXTRA_OPTS="${CARGO_BUILD_EXTRA_OPTS}"
.PHONY: build-riscv64-bin
build-riscv64-bin: ## Build greptime binary for riscv64 (linux-gnu) by the riscv64 cross dev-builder image.
docker run --network=host \
${ASSEMBLED_EXTRA_BUILD_ENV} \
-v ${PWD}:/greptimedb -v ${CARGO_REGISTRY_CACHE}:/root/.cargo/registry -v ${CARGO_GIT_CACHE}:/root/.cargo/git \
-w /greptimedb ${IMAGE_REGISTRY}/${IMAGE_NAMESPACE}/dev-builder-riscv64:${DEV_BUILDER_RISCV64_IMAGE_TAG} \
make build \
CARGO_EXTENSION="${CARGO_EXTENSION}" \
CARGO_PROFILE=${CARGO_PROFILE} \
FEATURES=${FEATURES} \
TARGET_DIR=${TARGET_DIR} \
TARGET=${RISCV64_TARGET} \
RELEASE=${RELEASE} \
CARGO_BUILD_EXTRA_OPTS="${CARGO_BUILD_EXTRA_OPTS}"
.PHONY: build-android-bin
build-android-bin: ## Build greptime binary for android.
docker run --network=host \
@@ -158,6 +176,14 @@ dev-builder: multi-platform-buildx ## Build dev-builder image.
-f docker/dev-builder/${BASE_IMAGE}/Dockerfile \
-t ${IMAGE_REGISTRY}/${IMAGE_NAMESPACE}/dev-builder-${BASE_IMAGE}:${DEV_BUILDER_IMAGE_TAG} ${BUILDX_MULTI_PLATFORM_BUILD_OPTS} .
.PHONY: dev-builder-riscv64
dev-builder-riscv64: ## Build the riscv64 cross dev-builder image.
docker build --network=host \
--build-arg="BASE_IMAGE=${DEV_BUILDER_RISCV64_BASE_IMAGE}" \
--build-arg="RUST_TOOLCHAIN=${RUST_TOOLCHAIN}" \
-f docker/dev-builder/riscv64/Dockerfile \
-t ${IMAGE_REGISTRY}/${IMAGE_NAMESPACE}/dev-builder-riscv64:${DEV_BUILDER_RISCV64_IMAGE_TAG} .
.PHONY: multi-platform-buildx
multi-platform-buildx: ## Create buildx multi-platform builder.
docker buildx inspect ${BUILDX_BUILDER_NAME} || docker buildx create --name ${BUILDX_BUILDER_NAME} --driver docker-container --bootstrap --use
+55
View File
@@ -0,0 +1,55 @@
# Cross-build environment for producing riscv64 (riscv64gc-unknown-linux-gnu)
# greptime binaries on x86_64/aarch64 hosts.
ARG BASE_IMAGE=ubuntu:22.04
FROM ${BASE_IMAGE}
ENV LANG=en_US.utf8
WORKDIR /greptimedb
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y software-properties-common
# Install dependencies, including the riscv64 cross toolchain.
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
libssl-dev \
tzdata \
curl \
unzip \
ca-certificates \
git \
build-essential \
pkg-config \
gcc-riscv64-linux-gnu \
g++-riscv64-linux-gnu
ARG TARGETPLATFORM
RUN echo "host platform: $TARGETPLATFORM"
ARG PROTOBUF_VERSION=29.3
# Install protobuf for the host arch (build scripts and protoc run on the host),
# because the one in the apt is too old (v3.12).
RUN if [ "$TARGETPLATFORM" = "linux/arm64" ]; then \
curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protoc-${PROTOBUF_VERSION}-linux-aarch_64.zip && \
unzip -o protoc-${PROTOBUF_VERSION}-linux-aarch_64.zip -d protoc3; \
else \
curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protoc-${PROTOBUF_VERSION}-linux-x86_64.zip && \
unzip -o protoc-${PROTOBUF_VERSION}-linux-x86_64.zip -d protoc3; \
fi
RUN mv -f protoc3/bin/* /usr/local/bin/
# Use cp -r instead of mv so that it also works when the base image already
# contains protobuf headers under /usr/local/include.
RUN cp -r protoc3/include/* /usr/local/include/
# Silence all `safe.directory` warnings, to avoid the "detect dubious repository"
# error when building with the repository mounted from a different user.
RUN git config --global --add safe.directory '*'
# Install Rust.
SHELL ["/bin/bash", "-c"]
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --no-modify-path --default-toolchain none -y
ENV PATH=/root/.cargo/bin/:$PATH
# Install Rust toolchains and the riscv64 target std.
ARG RUST_TOOLCHAIN
RUN rustup toolchain install ${RUST_TOOLCHAIN}
RUN rustup target add --toolchain ${RUST_TOOLCHAIN} riscv64gc-unknown-linux-gnu