ci: enhance benchmark workflows with version comparisons and diagnostics (#9383)

* ci: update Agent and Traces benchmark runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use unrestricted benchmark runtimes by default

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: retry benchmark image pulls before generating datasets

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: increase tracesbench defaults to 8 CPU and 32 GiB

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: configure tracesbench query timeout and update runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: update tracesbench runtime and configure import scan workers

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use shared parallel tracesbench import preparation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: update tracesbench runtime to preserve Tempo trace requests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: summarize tracesbench query results and lifecycle

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: show benchmark resource usage and update parallel loader runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(ci): resolve shared summary module in existing test harness

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: select tracesbench engines and GreptimeDB image version

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: update benchmark runtime with parallel Agent preparation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: select benchmark runtime registry by cloud provider

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: retry long-range benchmark image pulls

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: compare baseline and candidate GreptimeDB benchmarks

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: pin published dual-version benchmark runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: prepare agent benchmark indexes with updated runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: validate agent admission and render traces version comparison

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use runtime with traces index preparation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use consolidated runtime for benchmark version reports

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add license header to benchmark comparison tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(ci): read legacy Tempo ingestion metrics in traces summaries

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: require tracesbench query plan artifacts

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use optimized tracesbench report runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: update tracesbench runtime to merged report validation

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 6e1a137411)
Signed-off-by: WenyXu <wenymedia@gmail.com>
This commit is contained in:
Weny Xu
2026-10-12 12:49:21 +00:00
committed by WenyXu
parent 3796c478e2
commit f26e128b8a
13 changed files with 1187 additions and 41 deletions
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
# Pull only: never retry database writes, benchmarks, or cloud provisioning.
set -euo pipefail
if [[ $# != 1 || -z "$1" || "$1" == -* ]]; then
echo "Usage: $0 IMAGE" >&2
exit 2
fi
image=$1
attempts=3
for ((attempt=1; attempt<=attempts; attempt++)); do
echo "Pulling $image (attempt $attempt/$attempts)" >&2
if timeout --kill-after=10s 180s docker pull "$image"; then
exit 0
else
status=$?
fi
if ((attempt == attempts)); then
echo "Failed to pull $image after $attempts attempts (exit $status)" >&2
exit "$status"
fi
delay=$((attempt * 10))
echo "Pull failed (exit $status); retrying in ${delay}s" >&2
sleep "$delay"
done