mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-11-14 23:06:59 +00:00
discord9
4e024452fe
* fix(query): match the whole operand instead of copying matchers A native binary must report what Prometheus reports: when one side holds two series under one matching label set, the query fails with "found duplicate series for the match group ... on the right hand-side". The legacy route answers a value instead, because `matching_filters::propagate` copies the matcher into the other operand's scan and the groups without a partner are dropped before the join - its own documentation says that hides the duplicate. This branch had started applying that rewrite to native operands as well, and had refused multi-selector trees over partitioned tables in process to force the hybrid topology back. Both kept a legacy behaviour the native route does not need: a native binary reads independent raw sources and checks the whole one-side groups itself. Drop the rewrite for native operands, and keep the refusal only for several scans of one partitioned table, whose node cannot run inside a region. The partitioned acceptance case now asserts the duplicate error and that the pair really runs as one native binary; the legacy value stays in the test as the contrast that documents the difference. Signed-off-by: discord9 <discord9@163.com> * test(promql): assert the native binary, not the hybrid join The hybrid cases were written when a cross-table pair over a partitioned table could only run as two region-local producers joined on the frontend. The pair now runs as one native binary over independent raw sources, so those shape assertions describe a topology that no longer exists: two fused nodes, a frontend join, direct region scans under it and a projection above it. Assert what the native plan actually promises instead: the binary is one fused node with more than one input, every operand reads its own source, and the duplicate-group and value assertions stay as they are. The per-scan-partition value check now applies to operand producers only - a matching binary routes by label set, so an input partition may contribute nothing to a given output one. Signed-off-by: discord9 <discord9@163.com> * test(promql): keep the operand metrics for a native binary The per-scan-partition value check ran over every fused node, which now means the binary itself: it routes by label set, so an output partition may legitimately hold nothing from a given input partition. Check the two operand metrics instead - the same assertion the unpartitioned pair already makes - so each operand still proves it registered its values on every partition it reads. Signed-off-by: discord9 <discord9@163.com> * feat(query): admit multi-selector trees the native binary can match Three refusals were left over from when a multi-selector tree could only run inside one region over bare scans, and each of them now only costs coverage: - The aggregate split and the segmentation were admission authority: a tree that needed placement and did not fit the single-selector split fell back, even though each selector occurrence can read its own raw source and the consumer can run above the collection. They are optimizations now. - `multi_input` meant "the metric logical ids differ", so ordinary tables, repeated selectors and mixed engines were treated as a single input. It now means what the code needs: more than one independent input, one per selector occurrence, each bound to its own source. - Several scans of one partitioned table were refused in process. That node cannot run inside a region, but the same tree has a frontend placement over per-occurrence raw sources, so only the cross-table refusal that keeps the hybrid route is left. Tests that asserted the fallback now assert the native plan and the per-source inputs; the value, label and duplicate-group assertions are unchanged, and the shapes the compiler genuinely cannot express stay refused. Signed-off-by: discord9 <discord9@163.com> * fix(query): keep the raw sources under an in-process cut A table partitioned on a column that is not a tag splits one series across regions. When such a tree is cut into segments in process, the cut could fall back to the bare scan, and then a per-region prefix evaluated a window over one region's fragments: `rate` never saw the samples the other region holds, so a window that straddles the boundary produced no value at all. Remote execution already kept the raw source under the cut; in process the flag said otherwise. Pass the boundary fact that matters - the cut must keep each source when the table is split into regions - so the evaluated segment runs the prefix above the reconciliation. An unpartitioned table's scans are whole in one region, so its cuts keep the bare-scan form and the two-aggregate pairs still plan as one execution. The regression puts the two samples of one series on either side of a time-partition boundary and compares the native answer with the existing plan's for a two- and a three-selector window; without this change both lose the value. Signed-off-by: discord9 <discord9@163.com> * test(promql): prove the window runs natively The time-boundary regression compared rows with the existing plan, which a fallback also satisfies. Require a fused node in the frontend tree as well, so the test fails if the planner quietly declines the shape. Signed-off-by: discord9 <discord9@163.com> * test(query): trim redundant gates and assertions A review of this branch found duplications rather than missing coverage: two identical metric-name gates, a `!multi_input` term that now implies its own `selector.len() == 1` partner, a duplicate aggregate-admission test, a redundant cooperative-decorator special case, a tautological region-input assertion, an exact-child-type assertion that pins an intermediate shape, and a window regression that accepted a region-side fused node as proof of frontend admission. Remove them; the behaviour assertions stay. Signed-off-by: discord9 <discord9@163.com> * test(promql): require the native binary for a partitioned pair The values equal the legacy route's, so a legacy frontend join reading the same regions could satisfy the assertions on their own. Require the fused node in the frontend tree as well, and drop a duplicated rustdoc paragraph. Signed-off-by: discord9 <discord9@163.com> * fix(query): drop the stale unconditional canonical refusal A rebase left both the old refusal (`any_partitioned && contains_canonical_merge`) and the transport-scoped one, so the older block declined every canonical merge over a partitioned table on the wire and the acceptance case fell back to the existing planner. Keep the scoped refusal only. Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>
Setup tests for multiple storage backend
To run the integration test, please copy .env.example to .env in the project root folder and change the values on need.
Take s3 for example. You need to set your S3 bucket, access key id and secret key:
# Settings for s3 test
GT_S3_BUCKET=S3 bucket
GT_S3_REGION=S3 region
GT_S3_ACCESS_KEY_ID=S3 access key id
GT_S3_ACCESS_KEY=S3 secret access key
Run
Execute the following command in the project root folder:
cargo test integration
Test s3 storage:
cargo test s3
Test oss storage:
cargo test oss
Test azblob storage:
cargo test azblob
Setup tests with Kafka wal
To run the integration test, please copy .env.example to .env in the project root folder and change the values on need.
GT_KAFKA_ENDPOINTS = localhost:9092
Setup kafka standalone
cd tests-integration/fixtures
docker compose -f docker-compose.yml up kafka
Setup tests with etcd TLS
This guide explains how to set up and test TLS-enabled etcd connections in GreptimeDB integration tests.
Quick Start
TLS certificates are already at tests-integration/fixtures/etcd-tls-certs/.
-
Start TLS-enabled etcd:
cd tests-integration/fixtures docker compose up etcd-tls -d -
Start all services (including etcd-tls):
cd tests-integration/fixtures docker compose up -d --wait
Certificate Details
The checked-in certificates include:
ca.crt- Certificate Authority certificateserver.crt/server-key.pem- Server certificate for etcd-tls serviceclient.crt/client-key.pem- Client certificate for connecting to etcd-tls
The server certificate includes SANs for localhost, etcd-tls, 127.0.0.1, and ::1.
Regenerating Certificates (Optional)
If you need to regenerate the etcd certificates:
# Regenerate certificates (overwrites existing ones)
./scripts/generate-etcd-tls-certs.sh
# Or generate in custom location
./scripts/generate-etcd-tls-certs.sh /path/to/cert/directory
If you need to regenerate the mysql and postgres certificates:
# Regenerate certificates (overwrites existing ones)
./scripts/generate_certs.sh
# Or generate in custom location
./scripts/generate_certs.sh /path/to/cert/directory
Note: The checked-in certificates are for testing purposes only and should never be used in production.