mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-09-10 15:32:20 +00:00
localhost
4d65e8984a
* Implement `/query-regression` command handling and admission workflow - Add `query-regression-slash.py` script for processing `/query-regression` commands in PR comments, validating case arguments, and checking permissions. - Update `checks.yml` to include tests for the new slash command functionality. - Modify `query-regression-comment.yml` to trigger on the new `Query Regression Command` workflow. - Create `query-regression-slash.yml` to handle the dispatched command, validate allowlist and permissions, and initiate the regression workflow. - Enhance `query-regression.yml` to support additional inputs for PR admission and SHA verification. - Introduce `slash-command-dispatch.yml` to parse and dispatch commands from PR comments. - Document the new command admission process in `AGENTS.md` and `README.md`. - Add unit tests in `test_query_regression_slash.py` to cover command parsing and admission logic. * refactor: enhance query-regression command handling with comment validation and identity checks * feat: implement admission identity handling for query regression workflows * refactor: update PR admission logic in query regression workflow * refactor: update token usage in slash command dispatch and README for clarity * test: add cases for handling re-run failed jobs and stale runner artifacts * refactor: improve repository metadata handling in query regression scripts * chore: enable overwrite for artifact uploads to handle re-run failed jobs * chore: enable overwrite for query regression admission uploads * feat: enhance query-regression admission with HMAC signing and verification - Introduced HMAC signing for admission markers in query-regression workflows to ensure integrity and authenticity. - Updated `query-regression-comment.test.cjs` to include tests for signing and verifying admission markers. - Modified `query-regression-slash.py` to handle admission marker signing and verification, including checks for dispatch sender and head SHA consistency. - Enhanced workflows to securely manage admission markers and HMAC secrets, ensuring they are not exposed to untrusted contexts. - Improved documentation to clarify the admission process and the role of HMAC in securing the workflow. * test: add case to find newly posted marker among newer comments * test: add case to verify multiline output handling in write_outputs function
3.6 KiB
3.6 KiB
Agent Guidelines for Query Performance Tests
- Keep GitHub Actions YAML thin. Put non-trivial control flow, case expansion,
report generation, and metadata writing in scripts under
.github/scripts/; workflow steps should mostly invoke those scripts. - Runner lifecycle: the default path provisions one ephemeral Aliyun ECS
instance per run via
.github/scripts/aliyun-ecs-runner-provision.pyand always releases it viaaliyun-ecs-runner-teardown.py; a scheduled janitor workflow sweeps leftovers. Build caches live on that instance's system disk and are discarded with the VM. Runs do not share a workflow concurrency group. The ECS custom image is built from the runner Dockerfile by.github/runner-scale-sets/query-regression/ecs-image/build-ecs-image.py; keep the Dockerfile the single source of the tool contract. Dispatching with any otherrunnervalue treats it as a literal self-hosted runner label (seeecs-image/bootstrap-runner-host.shfor preparing such a host). - Query regression PR runs should build base/candidate binaries once, then run
the default case set. Do not hard-code a single case such as
promql_pushdown_7913into the workflow path. - Scheduled nightly comparison lives in
query-regression-nightly.yml: it waits for a successful Nightly Build, then callsquery-regression.ymlwith the previous vs current nightly SHAs. Keep SHA selection in.github/scripts/query-regression-nightly-refs.py. - PR comment admission is two workflows:
slash-command-dispatch.yml(peter-evans/slash-command-dispatch) decides whether a/commandshould run andrepository_dispatches payload context;query-regression-slash.ymlhandles/query-regression(allowlist, dispatcher head SHA, merge SHA, reusable call). Keep case-arg validation and dispatch-sender/head checks in.github/scripts/query-regression-slash.py. The admission job on ubuntu-latest posts a hidden HMAC-signed marker comment on the admitted PR and uploadsquery-regression-admissionas a lookup hint. The sticky-comment workflow verifies that marker (QUERY_REGRESSION_ADMISSION_HMAC, never referenced fromquery-regression.yml) before posting. There is no PR-label trigger. To add another command, list it in the dispatcher and add arepository_dispatchhandler. - The case DSL is not required to keep compatibility inside this PR. When the DSL changes, update TOML cases, the outer lifecycle script, Rust helpers, and docs together.
[case]is report metadata only.[scenario]is the executable regression configuration and must includekind, data layout, tables, queries, and thresholds. Rust owns case schema, defaults, validation, and normalized plan output throughquery_perf_fixture plan..github/scripts/query-regression-run.pyowns process lifecycle;query_regression_runnerconsumes normalized plans, frontend endpoints, direct-SST materialization requests, and OTLP target/finalize requests.- Keep the direct-SST generator generic. Issue-specific behavior belongs in case files and thresholds, not in Rust generator logic.
- Before pushing perf harness changes, run at least:
- the Python tests in the
test-toolingjob of.github/workflows/query-regression.yml(ubuntu-latest, not the ECS runner). The Checks workflow runs the same tests on ordinary PRs. cargo fmt --all -- --checkcargo build -p cmd --bin query_perf_fixture --features dev-toolscargo build -p cmd --bin query_regression_runner --features dev-tools- exercise the outer lifecycle script and Rust fixture generator against all built-in cases when the DSL or workflow case selection changes.
- the Python tests in the