mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-09-24 14:15:49 +00:00
Lei, HUANG
e9dd79d1d2
* fix(mito2): compare primary key ranges across schema versions Bind FileHandle ranges to the pinned region schema and append cached constant defaults to historical Dense keys. Preserve raw SST statistics, reject inexact bounds, and avoid invalidating views for unrelated metadata changes. Cover schema evolution, default changes, and tombstone retention through real compaction and reopen regressions. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor(mito2): report invalid primary key ranges Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor(mito2): scope primary key ranges to comparisons Keep raw PK bounds only in FileHandleInner and move schema-aware mapping and caching into task-local comparison contexts. Use explicit contexts for compaction overlap checks, window aggregation, and series scans. Preserve pinned-schema isolation, late statistics, and shared file lifecycle state without rebinding every handle. Cover cache isolation across region owners and adapt range fixtures to real Dense encodings. All 1530 mito2 tests and Clippy for all targets with the testing feature pass. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * refactor(mito2): cache aligned primary key ranges per file Replace task-local range maps with a single-slot cache in FileHandleInner, keyed by the target schema version. Preserve raw bounds for realignment across snapshots and default changes. Share schema mappers across comparison paths and use copy-on-write SST lists for metadata updates. Simplify range mapping to accept encoded bounds and assert the same-table contract at the file accessor. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * docs(mito2): clarify primary key mapper schema snapshot Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test(mito2): align primary key range fixtures with table contract Remove obsolete cross-table fallback expectations after region validation became a caller contract. Give compaction fixtures matching table identities, including the active-window L1 scenario. Clarify the mapper precondition and format the simplified alignment call. All 1530 mito2 tests and Clippy for all targets with the testing feature pass. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test(mito2): enable filesystem GC in release unit tests Let unit tests use the filesystem-backed object-store GC path regardless of optimization profile. Keep the production release GC selection unchanged. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * perf(mito-codec): skip release value decoding in PK prefix counts Validate field values only in debug builds and unit tests while keeping boundary, truncation, and trailing-byte checks in every build. Cover the linked library in debug and release integration tests, and verify that release unit tests still perform value validation. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * test(mito-codec): remove redundant prefix integration tests Retain the codec unit tests and cross-schema compaction regressions while dropping the standalone build-profile test file and its release-only invalid-value expectation. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> * ci: wait for MySQL to accept authenticated TCP queries Add a healthcheck using the configured test account and database. Docker Compose --wait previously only observed container startup because the fixture image had no healthcheck, allowing metasrv to connect before MySQL initialization completed. Verify readiness with SELECT 1 over TCP rather than the initialization socket. Signed-off-by: Lei, HUANG <ratuthomm@gmail.com> --------- Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Setup tests for multiple storage backend
To run the integration test, please copy .env.example to .env in the project root folder and change the values on need.
Take s3 for example. You need to set your S3 bucket, access key id and secret key:
# Settings for s3 test
GT_S3_BUCKET=S3 bucket
GT_S3_REGION=S3 region
GT_S3_ACCESS_KEY_ID=S3 access key id
GT_S3_ACCESS_KEY=S3 secret access key
Run
Execute the following command in the project root folder:
cargo test integration
Test s3 storage:
cargo test s3
Test oss storage:
cargo test oss
Test azblob storage:
cargo test azblob
Setup tests with Kafka wal
To run the integration test, please copy .env.example to .env in the project root folder and change the values on need.
GT_KAFKA_ENDPOINTS = localhost:9092
Setup kafka standalone
cd tests-integration/fixtures
docker compose -f docker-compose.yml up kafka
Setup tests with etcd TLS
This guide explains how to set up and test TLS-enabled etcd connections in GreptimeDB integration tests.
Quick Start
TLS certificates are already at tests-integration/fixtures/etcd-tls-certs/.
-
Start TLS-enabled etcd:
cd tests-integration/fixtures docker compose up etcd-tls -d -
Start all services (including etcd-tls):
cd tests-integration/fixtures docker compose up -d --wait
Certificate Details
The checked-in certificates include:
ca.crt- Certificate Authority certificateserver.crt/server-key.pem- Server certificate for etcd-tls serviceclient.crt/client-key.pem- Client certificate for connecting to etcd-tls
The server certificate includes SANs for localhost, etcd-tls, 127.0.0.1, and ::1.
Regenerating Certificates (Optional)
If you need to regenerate the etcd certificates:
# Regenerate certificates (overwrites existing ones)
./scripts/generate-etcd-tls-certs.sh
# Or generate in custom location
./scripts/generate-etcd-tls-certs.sh /path/to/cert/directory
If you need to regenerate the mysql and postgres certificates:
# Regenerate certificates (overwrites existing ones)
./scripts/generate_certs.sh
# Or generate in custom location
./scripts/generate_certs.sh /path/to/cert/directory
Note: The checked-in certificates are for testing purposes only and should never be used in production.