mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-07-03 20:40:37 +00:00
* fix: improve Grafana metrics dashboards (#8298) * chore: initial changes Signed-off-by: evenyag <realevenyag@gmail.com> * feat: improve troubleshooting dashboard Signed-off-by: evenyag <realevenyag@gmail.com> * chore: rm troubleshooting-dashboard.md Signed-off-by: evenyag <realevenyag@gmail.com> * chore: optimize metrics dashboard Signed-off-by: evenyag <realevenyag@gmail.com> * docs: move troubleshooting-dashboard.md Signed-off-by: evenyag <realevenyag@gmail.com> * chore: move mito gc duration panel Signed-off-by: evenyag <realevenyag@gmail.com> * chore: cleanup the dashboard - Overview trend panels are now aggregate-only: - Total Ingestion Rate Trend - Total Query Rate Trend - Protocol breakdowns remain in Ingestion and Queries. - Mito Backpressure and Failures no longer duplicates scan/GC signals. - Removed Write Stall per Instance. - Split Object Store and WAL into collapsed Object Store and collapsed WAL. - Moved WAL/logstore panels out of Storage into WAL. - Normalized OpenDAL “other request” matchers. - Normalized trigger elapsed p99/p75/avg aggregation. - Regenerated standalone JSON and dashboard YAML/Markdown. - Updated docs/troubleshooting-dashboard.md. Signed-off-by: evenyag <realevenyag@gmail.com> * fix: rearrange metasrv dashboard panels Signed-off-by: evenyag <realevenyag@gmail.com> * feat: improve troubleshooting dashboard layout Signed-off-by: evenyag <realevenyag@gmail.com> * docs: remove obsolete troubleshooting dashboard doc Signed-off-by: evenyag <realevenyag@gmail.com> * fix: correct cluster dashboard panel queries (missing _bucket, raw counters, rate normalization) Signed-off-by: evenyag <realevenyag@gmail.com> * fix: correct trigger panel datasource, collapse flush/compaction, split request latency panels Signed-off-by: evenyag <realevenyag@gmail.com> * fix: update grafana metrics dashboard panels Signed-off-by: evenyag <realevenyag@gmail.com> * fix: correct Grafana dashboard units Signed-off-by: evenyag <realevenyag@gmail.com> * chore: regenerate Grafana dashboards Signed-off-by: evenyag <realevenyag@gmail.com> * fix: use throughput unit for index IO bytes Signed-off-by: evenyag <realevenyag@gmail.com> --------- Signed-off-by: evenyag <realevenyag@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: redact Kafka SASL password in debug output (#8337) ## Summary - Mask `KafkaClientSaslConfig` password fields in debug output while keeping usernames visible. - Cover metasrv WAL debug output with a regression test. ## Files - `src/common/wal/src/config/kafka/common.rs` - `src/common/wal/src/config.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * fix(query): run optimizer rules before MergeScan (#8339) * fix(query): push down join filters before MergeScan Signed-off-by: discord9 <discord9@163.com> * fix(query): run optimizer before MergeScan pushdown Signed-off-by: discord9 <discord9@163.com> * fix(query): narrow pre-MergeScan filter pushdown Signed-off-by: discord9 <discord9@163.com> * fix(query): refine pre-MergeScan optimizer prepass Signed-off-by: discord9 <discord9@163.com> * fix(query): satisfy predicate extractor clippy Signed-off-by: discord9 <discord9@163.com> * test(query): cover pre-MergeScan optimizer edges Signed-off-by: discord9 <discord9@163.com> * test(query): cover set comparison prepass Signed-off-by: discord9 <discord9@163.com> * fix(query): guard remote scan filter pushdown Signed-off-by: discord9 <discord9@163.com> * fix(query): preserve subquery planning errors Signed-off-by: discord9 <discord9@163.com> * fix(query): preserve usable scan predicates Signed-off-by: discord9 <discord9@163.com> * fix(query): simplify scan predicate extraction Signed-off-by: discord9 <discord9@163.com> * fix(query): keep scan filter extraction scoped Signed-off-by: discord9 <discord9@163.com> * docs(query): explain pre-MergeScan optimizer Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: preserve bulk write grpc error details (#8349) Signed-off-by: jeremyhi <fengjiachun@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: include index files in GC listing (#8327) * fix: include index files in GC listing Signed-off-by: discord9 <discord9@163.com> * chore: filter GC index listing to puffins Signed-off-by: discord9 <discord9@163.com> * chore: simplify GC index listing stream Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: stream tables for prometheus label discovery (#8341) Signed-off-by: Ritwij Aryan Parmar <ritwij.aryan.parmar@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: account parquet metadata cache size (#8368) * fix: account parquet metadata cache size Use Parquet metadata memory sizing for SST metadata cache weight and add regression coverage for byte-array page-index buffers. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: saturate sst meta cache weight Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> --------- Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: respect gc mailbox timeout for admin gc (#8363) Signed-off-by: discord9 <discord9@163.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: record catalog and schema in slow queries (#8387) * fix: record catalog and schema in slow queries Add catalog and schema context to slow query records while appending the new columns after existing fields to preserve column order. - `src/common/frontend/src/slow_query_event.rs`: extend `SlowQueryEvent` schema and rows with `catalog_name` and `schema_name`, and cover append-only ordering. - `src/catalog/src/process_manager.rs`: carry catalog and schema through `SlowQueryTimer`. - `src/frontend/src/instance.rs`: capture context for SQL, plan, and PromQL slow query timers. - `tests-integration/tests/sql.rs`: assert MySQL and PostgreSQL slow query records include catalog and schema. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: address slow query review comment Use `String::clone` when writing slow query catalog and schema values. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: keep slow query schema only Remove the slow query `catalog_name` column and keep `schema_name` as a non-null tag dimension. - `src/common/frontend/src/slow_query_event.rs`: expose only `schema_name` in `SlowQueryEvent` rows and mark it as a tag. - `src/catalog/src/process_manager.rs`: stop carrying catalog context in `SlowQueryTimer`. - `src/frontend/src/instance.rs`: pass only schema context to slow query timers. - `tests-integration/tests/sql.rs`: assert slow query records include `schema_name` without `catalog_name`. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: schema name semantic should be field Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: typo Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: invalidate comment DDL cache and lock by object ID (#8390) * fix: invalidate comment ddl cache locally Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: fix typos Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: apply suggestions Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: client_ip error logs skip internal API (#8362) * chore: client_ip error logs skip internal API Signed-off-by: shuiyisong <xixing.sys@gmail.com> * fix: fmt Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: use const Signed-off-by: shuiyisong <xixing.sys@gmail.com> * chore: use const Signed-off-by: shuiyisong <xixing.sys@gmail.com> --------- Signed-off-by: shuiyisong <xixing.sys@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * feat: update dashboard to v0.13.6 (#8369) Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: use ENV for building dashboard (#8384) Signed-off-by: shuiyisong <xixing.sys@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: handle PromQL time binary aggregation (#8398) Signed-off-by: jeremyhi <fengjiachun@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * perf(mito): prune files by manifest time range (#8352) * perf(mito): prune files by manifest time range Signed-off-by: discord9 <discord9@163.com> * chore(mito): address file pruning review Signed-off-by: discord9 <discord9@163.com> * chore(mito): remove verbose file pruning log Signed-off-by: discord9 <discord9@163.com> * chore(mito): expose file pruning metric Signed-off-by: discord9 <discord9@163.com> * chore(mito): shorten file pruning metric Signed-off-by: discord9 <discord9@163.com> * test(mito): cover file pruning edge cases Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * perf(mito): skip manifest-pruned file ranges (#8366) * perf(mito): skip manifest-pruned file ranges Signed-off-by: discord9 <discord9@163.com> * test(mito): allow empty prune benchmark output Signed-off-by: discord9 <discord9@163.com> * fix(mito): avoid caching stale pruned builders Signed-off-by: discord9 <discord9@163.com> * chore(mito): address pruner clippy Signed-off-by: discord9 <discord9@163.com> * fix(mito): account worker pruner builder metrics Signed-off-by: discord9 <discord9@163.com> * test(mito): keep empty prune benchmark local Signed-off-by: discord9 <discord9@163.com> * refactor(mito): share manifest-pruned range skip Signed-off-by: discord9 <discord9@163.com> * chore(mito): shorten prune cache comment Signed-off-by: discord9 <discord9@163.com> * fix(mito): keep manifest prune state in pruner Signed-off-by: discord9 <discord9@163.com> * test(mito): cover manifest prune fast skip edge cases Signed-off-by: discord9 <discord9@163.com> * chore: fix typo in logical table alter Signed-off-by: discord9 <discord9@163.com> * chore(mito): address pruner review comments Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com> Signed-off-by: WenyXu <wenymedia@gmail.com> * chore: bump version to v1.1.2 Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: evenyag <realevenyag@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> Signed-off-by: discord9 <discord9@163.com> Signed-off-by: jeremyhi <fengjiachun@gmail.com> Signed-off-by: Ritwij Aryan Parmar <ritwij.aryan.parmar@gmail.com> Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: shuiyisong <xixing.sys@gmail.com> Co-authored-by: Yingwen <realevenyag@gmail.com> Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com> Co-authored-by: discord9 <discord9@163.com> Co-authored-by: jeremyhi <jiachun_feng@proton.me> Co-authored-by: Ritwij Aryan Parmar <88580521+RitwijParmar@users.noreply.github.com> Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com> Co-authored-by: sun <sunchang_long@163.com>
74 lines
2.7 KiB
Bash
Executable File
74 lines
2.7 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
|
|
# This script is used to download built dashboard assets from the "GreptimeTeam/dashboard" repository.
|
|
set -e
|
|
|
|
declare -r SCRIPT_DIR=$(cd $(dirname ${0}) >/dev/null 2>&1 && pwd)
|
|
declare -r ROOT_DIR=$(dirname ${SCRIPT_DIR})
|
|
declare -r STATIC_DIR="$ROOT_DIR/src/servers/dashboard"
|
|
OUT_DIR="${1:-$SCRIPT_DIR}"
|
|
|
|
DASHBOARD_REPOSITORY="${DASHBOARD_REPOSITORY:-GreptimeTeam/dashboard}"
|
|
DASHBOARD_ASSET="${DASHBOARD_ASSET:-build.tar.gz}"
|
|
DASHBOARD_GITHUB_TOKEN="${DASHBOARD_GITHUB_TOKEN:-${GH_TOKEN:-${GITHUB_TOKEN:-}}}"
|
|
RELEASE_VERSION="${DASHBOARD_RELEASE_VERSION:-$(cat "$STATIC_DIR/VERSION" | tr -d '\t\r\n ')}"
|
|
|
|
echo "Downloading assets to dir: $OUT_DIR"
|
|
cd $OUT_DIR
|
|
|
|
if [[ -z "$GITHUB_PROXY_URL" ]]; then
|
|
GITHUB_URL="https://github.com"
|
|
else
|
|
GITHUB_URL="${GITHUB_PROXY_URL%/}"
|
|
fi
|
|
|
|
function retry_fetch() {
|
|
local url=$1
|
|
local filename=$2
|
|
local auth_args=()
|
|
|
|
if [[ -n "$DASHBOARD_GITHUB_TOKEN" ]]; then
|
|
auth_args=(
|
|
-H "Authorization: Bearer ${DASHBOARD_GITHUB_TOKEN}"
|
|
-H "Accept: application/octet-stream"
|
|
)
|
|
fi
|
|
|
|
curl --connect-timeout 10 --retry 3 -fsSL "${auth_args[@]}" "$url" --output "$filename" || {
|
|
echo "Failed to download $url"
|
|
echo "You may try to set http_proxy and https_proxy environment variables."
|
|
if [[ -z "$GITHUB_PROXY_URL" ]]; then
|
|
echo "You may try to set GITHUB_PROXY_URL=http://mirror.ghproxy.com/https://github.com/"
|
|
fi
|
|
exit 1
|
|
}
|
|
}
|
|
|
|
# Download the SHA256 checksum attached to the release. To verify the integrity
|
|
# of the download, this checksum will be used to check the download tar file
|
|
# containing the built dashboard assets.
|
|
retry_fetch "${GITHUB_URL}/${DASHBOARD_REPOSITORY}/releases/download/${RELEASE_VERSION}/sha256.txt" sha256.txt
|
|
|
|
# Download the tar file containing the built dashboard assets.
|
|
retry_fetch "${GITHUB_URL}/${DASHBOARD_REPOSITORY}/releases/download/${RELEASE_VERSION}/${DASHBOARD_ASSET}" "$DASHBOARD_ASSET"
|
|
|
|
# Verify the checksums match; exit if they don't.
|
|
case "$(uname -s)" in
|
|
FreeBSD | Darwin)
|
|
echo "$(cat sha256.txt)" | shasum --algorithm 256 --check \
|
|
|| { echo "Checksums did not match for downloaded dashboard assets!"; exit 1; } ;;
|
|
Linux)
|
|
echo "$(cat sha256.txt)" | sha256sum --check -- \
|
|
|| { echo "Checksums did not match for downloaded dashboard assets!"; exit 1; } ;;
|
|
*)
|
|
echo "The '$(uname -s)' operating system is not supported as a build host for the dashboard" >&2
|
|
exit 1
|
|
esac
|
|
|
|
# Extract the assets and clean up.
|
|
tar -xzf "$DASHBOARD_ASSET" -C "$STATIC_DIR"
|
|
rm sha256.txt
|
|
rm "$DASHBOARD_ASSET"
|
|
|
|
echo "Successfully download dashboard assets to $STATIC_DIR"
|