mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-09-05 13:08:58 +00:00

Files

T

discord9 1693b2727c refactor: port query regression runner to Rust (#8651 )

* refactor: port query regression runner to Rust

Signed-off-by: discord9 <discord9@163.com>

* ci: remove optional OTLP report plotter

Signed-off-by: discord9 <discord9@163.com>

* refactor: split query regression runner into modules

Signed-off-by: discord9 <discord9@163.com>

* style: use crate-qualified imports in query regression runner

Signed-off-by: discord9 <discord9@163.com>

* refactor: simplify query regression runner internals

Signed-off-by: discord9 <discord9@163.com>

* feat: abstract inspect-footer storage access behind object store destination

Add an optional --destination <TOML> to inspect-footer (and
--base-destination/--candidate-destination to finalize-remote) so the
storage inspection reads DB data files through the opendal-backed
object_store abstraction instead of bare std::fs. Local paths keep
working unchanged via the --root shortcut (File backend); remote
backends (S3/GCS/...) are described by a DestinationConfig TOML
reusing the object-store crate's ObjectStoreConfig serde shape.

- inspect_footer: list via ObjectStore::list + ObjectMeta filtering
  (parquet keys, non-zero size, metadata/ segment), read footers
  async via ParquetObjectReader + ParquetMetaDataReader with known
  file size (no extra HEAD); output JSON schema unchanged
- finalize-remote: --base-data-home/--candidate-data-home become
  optional, mutually exclusive with the new --*-destination args
- cmd deps: add object_store_opendal + datafusion_object_store
- tests: fs-backend list+footer integration tests (metadata filtering,
  destination TOML mode, root/destination exclusivity)

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* style: drop needless borrow in inspect footer test

Fix clippy::needless_borrows_for_generic_args in the inspect-footer test
(fs::create_dir_all(table.join("metadata"))). Missed by the earlier
focused clippy run because it only covered --bin targets.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

2026-08-06 13:01:32 +00:00

2.0 KiB

Raw Blame History

Agent Guidelines for Query Performance Tests

Keep GitHub Actions YAML thin. Put non-trivial control flow, case expansion, report generation, and metadata writing in scripts under .github/scripts/; workflow steps should mostly invoke those scripts.
Query regression PR runs should build base/candidate binaries once, then run the default case set. Do not hard-code a single case such as promql_pushdown_7913 into the workflow path.
The case DSL is not required to keep compatibility inside this PR. When the DSL changes, update TOML cases, the outer lifecycle script, Rust helpers, and docs together.
[case] is report metadata only. [scenario] is the executable regression configuration and must include kind, data layout, tables, queries, and thresholds. Rust owns case schema, defaults, validation, and normalized plan output through query_perf_fixture plan. .github/scripts/query-regression-run.py owns process lifecycle; query_regression_runner consumes normalized plans, frontend endpoints, direct-SST materialization requests, and OTLP target/finalize requests.
Keep the direct-SST generator generic. Issue-specific behavior belongs in case files and thresholds, not in Rust generator logic.
Before pushing perf harness changes, run at least:
- uv run --no-project python -m py_compile .github/scripts/query-regression-run.py .github/scripts/query-regression-summary.py .github/scripts/query-regression-pr-metadata.py tests/perf/test_query_regression_runner_compaction_toctou.py tests/perf/test_query_regression_runner_otlp_trace_load.py
- uv run --no-project python tests/perf/test_query_regression_runner_compaction_toctou.py && uv run --no-project python tests/perf/test_query_regression_runner_otlp_trace_load.py
- cargo fmt --all -- --check
- cargo build -p cmd --bin query_perf_fixture --features dev-tools
- exercise the outer lifecycle script and Rust fixture generator against all built-in cases when the DSL or workflow case selection changes.

2.0 KiB Raw Blame History

Agent Guidelines for Query Performance Tests

2.0 KiB

Raw Blame History