Files
greptimedb/tests-integration
discord9 b35967ca0b feat(promql): answer subquery matrix roots natively (#10095)
* feat(promql): answer a subquery matrix root on the native route

A matrix-valued instant query whose root is a subquery (`m[5m:1m]`, with `offset` and `@`)
now runs natively: the planner recognises the root, derives the subquery's child grid and
compiles the inner expression on it, while the statement's own bounds stay where `@ start()`
and `@ end()` resolve against them. No window node is added - the tree is the inner
expression, and its rows carry the evaluation timestamps, which is what the HTTP encoder
reports as `resultType: matrix`.

A raw range selector still declines (raw samples have no evaluation grid to bind), and a
range query with a matrix root declines so the existing planner's "range vector" rejection is
what the caller sees. A pre-epoch statement (`TQL EVAL ('-60', '-60', '1m')`) rebuilds its
bounds on the side of the epoch their sign names, and a statement whose grid cannot be built
keeps the existing plan.

`compile_statement_with_grid` / `compile_selectors_with_grid` take the grid explicitly; the
grid is still derived lazily inside the compiler so an unsupported statement reports the
reason it has no tree, not a grid complaint.

Verified with `-p query`, `-p promql`, `tests-integration --lib` (the matrix-root cases) and
the file-backed HTTP route test.

Signed-off-by: discord9 <discord9@163.com>

* fix(promql): bind the subquery's child grid on every lowering path

`compile_statement_with_grid` compiled a subquery root's tree on the child grid and the plain
`FusedSeries::try_new` path bound that grid, but the split-aggregate, canonical-boundary and segment
paths still received the statement grid. A matrix root whose inner expression folds as an aggregate
therefore built a final node publishing on the statement grid over a tree compiled on the child grid,
and `plan_split_aggregate` refused it with `FusedSeries output is not on the statement grid`:
`sum(pm)[5m:1m]`, `(sum by (job) (rate(pm[2m])))[5m:1m]`, its `@ 600` form and its `* 2` form all
failed instead of answering the five child-grid points.

The effective grid is now chosen once, right after the statement grid is known - the child grid for a
subquery root, the statement grid otherwise - and every lowering path binds that one. The per-site
`unwrap_or_else` workarounds and `fused_publish_grid` go away with it: they were compensating for two
halves that disagreed, and the binding is now the grid the tree was compiled on.

The child-grid derivation itself is back in one function: `subquery_child_grid` in
`native_compile.rs` feeds both the compiler (which binds it into the window node) and the planner
(which binds the rows to it), using the checked `range_millis`/`millis` conversions and the compiler's
zero-step refusal. The planner's inline copy used unchecked `as i64` and its own default step, so an
overflowing range wrapped there while the compiler reported it.

`native_promql_matrix_root_matches_legacy` covers the four shapes, asserting both the route and the
child-grid timestamps for the split case.

Signed-off-by: discord9 <discord9@163.com>

* test(promql): cover pre-epoch non-matrix routing

Signed-off-by: discord9 <discord9@163.com>

* fix(promql): publish recursive segments on their declared grid

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-11-09 22:37:23 +08:00
..

Setup tests for multiple storage backend

To run the integration test, please copy .env.example to .env in the project root folder and change the values on need.

Take s3 for example. You need to set your S3 bucket, access key id and secret key:

# Settings for s3 test
GT_S3_BUCKET=S3 bucket
GT_S3_REGION=S3 region
GT_S3_ACCESS_KEY_ID=S3 access key id
GT_S3_ACCESS_KEY=S3 secret access key

Run

Execute the following command in the project root folder:

cargo test integration

Test s3 storage:

cargo test s3

Test oss storage:

cargo test oss

Test azblob storage:

cargo test azblob

Setup tests with Kafka wal

To run the integration test, please copy .env.example to .env in the project root folder and change the values on need.

GT_KAFKA_ENDPOINTS = localhost:9092

Setup kafka standalone

cd tests-integration/fixtures

docker compose -f docker-compose.yml up kafka

Setup tests with etcd TLS

This guide explains how to set up and test TLS-enabled etcd connections in GreptimeDB integration tests.

Quick Start

TLS certificates are already at tests-integration/fixtures/etcd-tls-certs/.

  1. Start TLS-enabled etcd:

    cd tests-integration/fixtures
    docker compose up etcd-tls -d
    
  2. Start all services (including etcd-tls):

    cd tests-integration/fixtures
    docker compose up -d --wait
    

Certificate Details

The checked-in certificates include:

  • ca.crt - Certificate Authority certificate
  • server.crt / server-key.pem - Server certificate for etcd-tls service
  • client.crt / client-key.pem - Client certificate for connecting to etcd-tls

The server certificate includes SANs for localhost, etcd-tls, 127.0.0.1, and ::1.

Regenerating Certificates (Optional)

If you need to regenerate the etcd certificates:

# Regenerate certificates (overwrites existing ones)
./scripts/generate-etcd-tls-certs.sh

# Or generate in custom location
./scripts/generate-etcd-tls-certs.sh /path/to/cert/directory

If you need to regenerate the mysql and postgres certificates:

# Regenerate certificates (overwrites existing ones)
./scripts/generate_certs.sh

# Or generate in custom location
./scripts/generate_certs.sh /path/to/cert/directory

Note: The checked-in certificates are for testing purposes only and should never be used in production.