mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-10-26 21:56:17 +00:00
393c68b101
* fix(meta): populate physical metric table column ids (#9286) * fix(meta): populate physical metric table column ids Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> * test(meta): verify physical metric column ids Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> --------- Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> (cherry picked from commitf1e9a74f00) Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix: preserve primary key order when syncing columns (#9189) * fix: preserve primary key order when syncing columns Signed-off-by: WenyXu <wenymedia@gmail.com> * fix: reject invalid SyncColumns metadata Signed-off-by: WenyXu <wenymedia@gmail.com> --------- Signed-off-by: WenyXu <wenymedia@gmail.com> (cherry picked from commit5ffd01a70a) Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(mito2): truncate parquet column index min/max for SST writes (#9420) * fix(mito2): truncate parquet column index min/max for SST writes The SST writer and the bulk part encoder disabled column index truncation together with statistics truncation in #6977. Mito never reads the column index, but every page of a large string or binary column still stored its full min and max values, uncompressed. Truncate the column index to the parquet default of 64 bytes and keep chunk statistics untruncated, since primary key pruning decodes them. Signed-off-by: Dennis Zhuang <killme2008@gmail.com> * docs(mito2): note that the column index must not be used to decode primary keys Signed-off-by: Dennis Zhuang <killme2008@gmail.com> --------- Signed-off-by: Dennis Zhuang <killme2008@gmail.com> (cherry picked from commita314ac4281) Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(partition): preserve region mapping during repartition (#9458) Signed-off-by: Dennis Zhuang <killme2008@gmail.com> (cherry picked from commit90f117b957) Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(otlp): convert boolean metric attributes into label values (#9468) Signed-off-by: fengyue-xve <1759519137@qq.com> (cherry picked from commit2ebb03fbf6) Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(metasrv): stop parking the heartbeat response on the region supervisor queue (#9474) * fix(meta-srv): stop parking the heartbeat response on the region supervisor queue Signed-off-by: lczllx <2181719471@qq.com> * test(meta-srv): cover the heartbeat response path at the handler level Signed-off-by: lczllx <2181719471@qq.com> --------- Signed-off-by: lczllx <2181719471@qq.com> (cherry picked from commitd1151b0ed6) Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * fix(promql): stop fabricating histogram fold hash partitioning (#9509) * fix(promql): stop fabricating histogram fold hash partitioning Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test(promql): cover histogram fraction binary join missing rows Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test(promql): normalize dictionary labels in histogram regression Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: cover histogram joins on logical metric tables Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: specify insert columns for logical metric fixture Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test: use supported histogram child replacement API Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * test(promql): keep histogram fold fixtures truthful Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> --------- Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> (cherry picked from commit2039f508fc) Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test: use with_new_children in histogram fold partitioning test Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test: avoid group_left label semantics in histogram fold join test Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test: import ParquetRecordBatchReaderBuilder for column index truncation test Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test: write enough rows to flush several pages with parquet 58 Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com> Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> Signed-off-by: WenyXu <wenymedia@gmail.com> Signed-off-by: Dennis Zhuang <killme2008@gmail.com> Signed-off-by: fengyue-xve <1759519137@qq.com> Signed-off-by: lczllx <2181719471@qq.com> Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Co-authored-by: Dhruv Vaishnav <dhruvvaishnav687@gmail.com> Co-authored-by: Weny Xu <wenymedia@gmail.com> Co-authored-by: dennis zhuang <killme2008@gmail.com> Co-authored-by: fengyue-xve <1759519137@qq.com> Co-authored-by: Dragon Roar <2181719471@qq.com> Co-authored-by: discord9 <discord9@163.com>
Setup tests for multiple storage backend
To run the integration test, please copy .env.example to .env in the project root folder and change the values on need.
Take s3 for example. You need to set your S3 bucket, access key id and secret key:
# Settings for s3 test
GT_S3_BUCKET=S3 bucket
GT_S3_REGION=S3 region
GT_S3_ACCESS_KEY_ID=S3 access key id
GT_S3_ACCESS_KEY=S3 secret access key
Run
Execute the following command in the project root folder:
cargo test integration
Test s3 storage:
cargo test s3
Test oss storage:
cargo test oss
Test azblob storage:
cargo test azblob
Setup tests with Kafka wal
To run the integration test, please copy .env.example to .env in the project root folder and change the values on need.
GT_KAFKA_ENDPOINTS = localhost:9092
Setup kafka standalone
cd tests-integration/fixtures
docker compose -f docker-compose.yml up kafka
Setup tests with etcd TLS
This guide explains how to set up and test TLS-enabled etcd connections in GreptimeDB integration tests.
Quick Start
TLS certificates are already at tests-integration/fixtures/etcd-tls-certs/.
-
Start TLS-enabled etcd:
cd tests-integration/fixtures docker compose up etcd-tls -d -
Start all services (including etcd-tls):
cd tests-integration/fixtures docker compose up -d --wait
Certificate Details
The checked-in certificates include:
ca.crt- Certificate Authority certificateserver.crt/server-key.pem- Server certificate for etcd-tls serviceclient.crt/client-key.pem- Client certificate for connecting to etcd-tls
The server certificate includes SANs for localhost, etcd-tls, 127.0.0.1, and ::1.
Regenerating Certificates (Optional)
If you need to regenerate the etcd certificates:
# Regenerate certificates (overwrites existing ones)
./scripts/generate-etcd-tls-certs.sh
# Or generate in custom location
./scripts/generate-etcd-tls-certs.sh /path/to/cert/directory
If you need to regenerate the mysql and postgres certificates:
# Regenerate certificates (overwrites existing ones)
./scripts/generate_certs.sh
# Or generate in custom location
./scripts/generate_certs.sh /path/to/cert/directory
Note: The checked-in certificates are for testing purposes only and should never be used in production.