* refactor(promql): read the legacy histogram estimators from the native kernel `prom_native_histogram_quantile` and `prom_native_histogram_fraction` estimated from `common_query`'s `NativeHistogram` bucket walk: exponential bucket bounds computed as `2^x`, and a forward walk from the lowest bucket only. The native kernel in `histogram_eval` walks the same histogram the way Prometheus does -- the bound table in `bounds`, and a walk from the highest bucket down once a finite-sum quantile reaches the median -- so one histogram had two answers that differ in their last places. Read the `NativeHistogram` through the borrowed `HistogramView` both representations share and return the kernel's answer, mapping its outcome back onto the NaN-observation notes the legacy UDFs record. Their signatures and the notes are unchanged; only the arithmetic behind them moves. The `2^x` estimator in `common_query::native_histogram` stays for that crate's public API and is annotated as such: no PromQL path reaches it any more. Signed-off-by: discord9 <discord9@163.com> * refactor(promql): delegate the legacy histogram arithmetic to the native kernel `prom_native_histogram_add`, `_sub`, `_mul_scalar`, `_scalar_mul`, `_div_scalar`, `_neg`, the sum/avg accumulators, `delta`/`idelta`/`rate`/`increase`, and the weighted mean that merges partial states all scaled and combined `NativeHistogram`s with that type's own reconciliation: its own zero-bucket widening, schema reduction and custom-bound intersection, over `2^x` bounds. `FloatHistogram::add`/`sub`/`mul`/`div` already implement Prometheus's rules (`addBuckets`, `reconcileZeroBuckets`, `CopyToSchema`, `intersectCustomBucketBounds`) and the native kernel's golden tests cover them, so this keeps one implementation of the semantics. `AddOutcome` folds into the existing semantics rather than being dropped: `custom_bounds_reconciled` is the intersection the kernel keeps in `custom_values`, which the callers already announce from the operands, and `counter_reset_collision` is the contradiction the kernel answers with the unknown hint the legacy arithmetic derived from two hints that disagree; both are asserted in debug builds. The public `NativeHistogram` signatures are unchanged, the sums are compacted as before so `promql_eq` and the serialized spans keep their answers, and unary minus goes through the kernel's multiplication like the other scalings. Signed-off-by: discord9 <discord9@163.com> * test(promql): pin the legacy delegation's observable answers Comparing the bridge against the kernel it calls only shows they agree today; it cannot show what either answer means. These tests pin the answers themselves. - quantile with a NaN sum over several buckets answers 32.0 with NaNSkew, because the NaN walk leaves the last bucket for the interpolation; below the rank it answers NaN with NaNResult. - an exact tie answers 1.0 through the legacy walk and 2.0 through the kernel, on a fixture with an empty bucket between the two, and neither records a note. - assert_same_payload now compares every field, including the start timestamp policy, custom values, the complete span segmentation and the bucket bits. - round-tripping preserves reset hints, custom values, zero-length spans and gaps. - add/sub reconcile the reset hint and the custom bounds, compact cancelled buckets, and keep promql_eq's expanded-index equality. - scale/divide_by keep the field-by-field fallback for an unreadable layout -- mismatched counts, a backwards span, an overflowing index, a negative length -- and preserve its start timestamp; add/sub answer None for the same layouts. Zero, negative and NaN factors are pinned. - rate/increase/delta/idelta/irate clear the start timestamp of a derived histogram, while a single-sample sum aggregate keeps its stored one. A layout whose later span steps backwards is refused before the kernel sees it: the legacy bucket_indices rejected it, while side_from_view would have accepted repeated indices. Signed-off-by: discord9 <discord9@163.com> * test(http): pin the legacy histogram route's rate, NaN quantile and derived windows The delegation changes what three kinds of well-formed data answer, so assert it where a user sees it. Every request runs without the native hint and asserts !executes_fused_series on the observed plan, which proves the legacy route rather than assuming it from the missing hint. - rate and increase over a raw sample with a start timestamp inside the window still extrapolate from the synthetic zero: count 8/3 and sum 10/3 for a rate, 8 and 10 for an increase, with no infos or warnings. - histogram_quantile(0.5) over schema 0, buckets [8, 1], count 10 and a NaN sum answers 32.0 and reports the NaN-observation info, matching Prometheus's walk that leaves the last bucket for the interpolation. The previous estimator answered 2^-0.375. - rate over a raw window answers one series while the same window over h*1 answers none: the derived sample carries no start timestamp, so it has no synthetic zero to extrapolate from. Both phases are covered, memtable and flushed SST. The native-histogram test now installs the route interceptor the way its sibling tests do, so the plan can be observed; all of its original assertions are unchanged. Signed-off-by: discord9 <discord9@163.com> --------- Signed-off-by: discord9 <discord9@163.com>
Setup tests for multiple storage backend
To run the integration test, please copy .env.example to .env in the project root folder and change the values on need.
Take s3 for example. You need to set your S3 bucket, access key id and secret key:
# Settings for s3 test
GT_S3_BUCKET=S3 bucket
GT_S3_REGION=S3 region
GT_S3_ACCESS_KEY_ID=S3 access key id
GT_S3_ACCESS_KEY=S3 secret access key
Run
Execute the following command in the project root folder:
cargo test integration
Test s3 storage:
cargo test s3
Test oss storage:
cargo test oss
Test azblob storage:
cargo test azblob
Setup tests with Kafka wal
To run the integration test, please copy .env.example to .env in the project root folder and change the values on need.
GT_KAFKA_ENDPOINTS = localhost:9092
Setup kafka standalone
cd tests-integration/fixtures
docker compose -f docker-compose.yml up kafka
Setup tests with etcd TLS
This guide explains how to set up and test TLS-enabled etcd connections in GreptimeDB integration tests.
Quick Start
TLS certificates are already at tests-integration/fixtures/etcd-tls-certs/.
-
Start TLS-enabled etcd:
cd tests-integration/fixtures docker compose up etcd-tls -d -
Start all services (including etcd-tls):
cd tests-integration/fixtures docker compose up -d --wait
Certificate Details
The checked-in certificates include:
ca.crt- Certificate Authority certificateserver.crt/server-key.pem- Server certificate for etcd-tls serviceclient.crt/client-key.pem- Client certificate for connecting to etcd-tls
The server certificate includes SANs for localhost, etcd-tls, 127.0.0.1, and ::1.
Regenerating Certificates (Optional)
If you need to regenerate the etcd certificates:
# Regenerate certificates (overwrites existing ones)
./scripts/generate-etcd-tls-certs.sh
# Or generate in custom location
./scripts/generate-etcd-tls-certs.sh /path/to/cert/directory
If you need to regenerate the mysql and postgres certificates:
# Regenerate certificates (overwrites existing ones)
./scripts/generate_certs.sh
# Or generate in custom location
./scripts/generate_certs.sh /path/to/cert/directory
Note: The checked-in certificates are for testing purposes only and should never be used in production.