2b05da4c06 fix: backport suggested independent fixes to release/5e02efa9 (#9795)
* fix(servers): resolve port from IPv6 bind addresses (#9142)

Signed-off-by: immanuwell <pchpr.00@list.ru>
(cherry picked from commit 5ec679fed9)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: register region migration procedures before returning IDs (#9163)

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit c65e8d0962)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(promql): skip empty batches in SeriesDivide stream (#9178)

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 5e57b00a95)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(servers): merge duplicate timeseries across RecordBatches in Prometheus remote read (#9032)

* fix(servers): merge timeseries across RecordBatches in Prometheus remote read

Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>

* fix(servers): exercise recordbatches_to_timeseries and sort samples with sort_unstable_by_key

---------

Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
(cherry picked from commit 59eb63c663)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(prometheus): honor label matchers in __name__ values query (#9134)

* fix(prometheus): honor label matchers in __name__ values query

`/api/v1/label/__name__/values?match[]={pod="abc"}` dropped every matcher
other than `__name__` and returned all metrics in the schema. No error,
just the wrong list. Grafana's metrics browser sends this request, so
picking a label value there did nothing.

Selectors that only constrain `__name__` keep answering from table
metadata. A selector constraining an ordinary label now goes to the data:
scan each metric engine physical table for distinct `__table_id` in the
time range, map the ids back to metric names, then apply the selector's
own `__name__` matchers.

Only metric engine tables are covered; other engines share no column space
to scan.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor(prometheus): batch-resolve metric names by table id

Building a full table-id-to-name map meant walking every table in the
schema and holding all of them in memory, just to name the handful the
scan returned. Use `tables_by_ids` instead — one batch KV read over the
ids the scan actually produced.

The catalog walk stays, but only to find the physical tables to scan.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): read an absent label as the empty string

A matcher on a label the series does not carry only worked when the table
had no column for it at all. Where the column exists but is NULL on that
row -- the norm for logical metrics sharing a metric engine physical
table, which holds the union of their label columns -- three-valued logic
dropped the row, so `host!="host1"` and `host=""` missed every metric
without a host label.

Coalesce nullable string label columns to "" for matchers that accept the
empty string, rather than only for the OTLP temporality marker. Equality
matchers are untouched; they cannot match NULL either way.

This is the Prometheus compatibility fix #8970 deliberately kept out of
its own scope. The cost is visible in the regex sqlness plan: the
predicate becomes a CASE, so the scan loses its LastRow selector and
grows a FilterExec. Only negative and empty-accepting matchers pay it.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): don't panic on pre-epoch label value bounds

`rewrite_label_values_query` unwrapped `duration_since(UNIX_EPOCH)`, which
returns an error for an instant before the epoch. `start=1969-12-31T23:59:59Z`
parses as valid RFC3339, so the request panicked instead of answering.

Recover the sign from the error branch, and report a value beyond i64
milliseconds as an error rather than wrapping the cast.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor(prometheus): drop applicable_matchers, share the distinct scan

With the planner reading an absent label as empty, the frontend no longer
needs to pre-filter matchers per physical table. Removing that exposed a
second problem: a physical table that never took a column from a logical
table exposes no `__table_id`, and projecting it failed the whole request.
Skip those tables; the only thing that can miss is a metric with no labels.

Also pulls out the plan-build-execute-collect sequence the two label value
scans had in common.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 007836ba7a)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(mito2): cancel cache construction for incomplete scans (#9254)

* fix(mito2): cancel cache construction for incomplete scans

CacheBatchBuffer spawned the background concat task and dropped its join
handle. A scan that was cancelled or failed therefore left the task alive:
it kept compacting already queued batches, and while waiting for a range
result memory permit it held them, even though without a finish command
the result can never be put into the cache.

Keep the handle and abort it when the buffer is dropped. The handle is
cleared once the task owns the finish command, so a completed scan still
populates the cache after its stream is dropped. Abort does not preempt a
concat that is already running; it takes effect the next time the task is
polled.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(mito2): wait for the permit park before cancelling the buffer

An empty buffered_batches only proves the batches were enqueued, so the
cancellation test could abort a concat task that had never been polled.
Count acquisitions that find too few permits, a test-only signal, and drop
the buffer once the task has reached that wait. Nothing awaits between the
check and the parking, so an observed increment means the caller is about
to wait for permits the test holds for the rest of the case.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 43ce07fa69)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: merge every state row in geo_path and json_encode_path (#9339)

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit cc1be82858)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: align ordered aggregate state type with the accumulator output (#9340)

* fix: align ordered aggregate state type with the accumulator output

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: reject mismatched aggregate states and keep hard-ordered aggregates unsplit

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: keep WITHIN GROUP aggregates splittable and relabel state fields by position

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit affc0a1b1d)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(mito2): stop parallel flat scan tasks once the receiver is dropped (#9368)

* fix(mito2): stop parallel flat scan tasks once the receiver is dropped

spawn_flat_scan_task ignored send errors, so after a query was cancelled the
detached task kept reading its source to EOF. Break out of the loop when the
receiver is gone.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore(mito2): log when a parallel scan task stops on a dropped receiver

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore(mito2): include the send error in the parallel scan stop log

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 411c0f173e)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(catalog): skip backend batch_get on full cache hits (#9431)

fix(catalog): return cached batch values without an empty backend call

Count backend batch_get invocations in regression tests and cover empty input.

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit acebd6b35b)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: coalesce equal histogram quantile bucket boundaries (#9451)

* fix: coalesce equal histogram quantile bucket boundaries

Merge numerically equivalent bucket boundaries before monotonicity repair, preserving the allocation-free path for unique boundaries. Share coalescing with histogram_fraction and cover duplicate boundaries with unit and SQL regressions.

Fixes #9443

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: keep duplicate infinity buckets in one histogram group

Confirm classic histogram completion at a series/timestamp boundary or EOF instead of the first positive infinity. Retain lookahead across batches and reject incomplete optimistic groups so equivalent infinity labels reach numeric coalescing together.

Add stream and SQL regressions for duplicate infinity labels, including established bucket counts and EOF. Correct older fixtures to use distinct keys for distinct histogram groups.

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: avoid buffering histogram groups with invalid trailing bounds

Discover candidate bucket counts at the first group boundary and let optimistic validation fall back to safe mode for malformed bounds. Document discovery and cover trailing invalid and null labels with a focused streaming regression.

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit 97913b4b3a)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(otlp): coerce empty and "-" trace attribute strings to null (#9461)

* fix(otlp): coerce empty and "-" trace attribute strings to null

Instrumentations fill numeric attributes from absent headers with an empty
string (PSR-7 getHeaderLine returns ""). Coercing that into an existing
numeric column failed and rejected the whole span.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: trim redundant row from empty-string rewrite test

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit d640ab8bc5)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(metasrv): retry failed heartbeat address updates (#9437)

* fix(metasrv): retry failed heartbeat address updates

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(metasrv): move shared heartbeat helpers to utils

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 65b9a35692)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: align TWCS assignment with half-open time windows (#9464)

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
(cherry picked from commit 5126623e63)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(promql): keep vector labels when a computed scalar is on the right (#9478)

* fix(promql): keep vector labels when a computed scalar is on the right

`metric + time()` or `metric * scalar(other)` planned the join with the
scalar side's context, which has no tags, so every series collapsed into
one unlabeled series. Use the vector side's context when the right operand
is scalar-typed.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): update results for labels kept with a computed scalar on the right

v * pi() and v + time() now keep the vector's labels and time index name,
as in Prometheus.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): qualify columns of an unnamed operand in vector joins

A comparison filter's projection has no table name, so after the join its
time index resolved unqualified and was ambiguous when the other operand had
the same column. Also keep the empty-metric branch unchanged for scalar
left operands.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): cover comparison filters joined with a scalar

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 6da5352c53)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(promql): keep metric name for comparison operators (#9332)

* fix(promql): keep metric name for comparison operators

`collect_metric_names` dropped the response `__name__` for every binary
operator except `and` / `or` / `unless`, so comparisons such as
`node_load1 > 5` came back without the metric name. Grafana rendered them
as `{instance="..."}` and alert templates reading `$labels.__name__` got
nothing.

Follow Prometheus' `shouldDropMetricName` / `resultMetric`: arithmetic
operators drop the name, comparisons keep the left-hand name, except when
the comparison returns a bool or is a one-to-one `on(...)` match. A
`group_left` / `group_right` modifier does not reduce the labels, so the
name is kept there.

Both the HTTP PromQL API and the gRPC gateway share
`retrieve_metric_name_and_result_type`, so one fix covers both.

Signed-off-by: Aumnertic <hznuzc@gmail.com>

* docs: document binary_keeps_metric_name

Signed-off-by: Aumnertic <hznuzc@gmail.com>

* fix(promql): handle on(__name__), scalar lhs and group_right

Signed-off-by: Aumnertic <hznuzc@gmail.com>

* chore(promql): correct result-side docs and drop unreachable String arm

Signed-off-by: Aumnertic <hznuzc@gmail.com>

* fix(promql): drop metric name when explicitly ignored

Handle ignoring(__name__) for one-to-one comparisons and cover
comparison, group_left, group_right and unless behavior.

Signed-off-by: Aumnertic <hznuzc@gmail.com>

---------

Signed-off-by: Aumnertic <hznuzc@gmail.com>
(cherry picked from commit c33a6cfa56)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(query): keep filter fetch when rewriting matches term (#9521)

* test: cover matches term limit dropped by constant rewrite

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(query): keep filter fetch when rewriting matches term

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 483d1e49a3)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* style: format #9437 test imports for the release toolchain

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test: regenerate array_agg result for release plan rendering

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: immanuwell <pchpr.00@list.ru>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: Aumnertic <hznuzc@gmail.com>
Co-authored-by: Immanuel Tikhonov <122638311+immanuwell@users.noreply.github.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: Mohd Quamar Tyagi <104281681+Tyagiquamar@users.noreply.github.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: Aum <hznuzc@gmail.com>
2026-10-20 09:46:46 -07:00
2023-08-10 08:08:37 +00:00
2023-06-25 11:05:46 +08:00
2023-11-09 10:38:12 +00:00
2023-03-28 19:14:29 +08:00

GreptimeDB Logo

Metrics, logs, and traces.
One engine, on your infrastructure.

A columnar database for metrics, logs, and traces on object storage. Apache-2.0 licensed core.

User Guide | API Docs | Roadmap 2026

Stable Canary Nightly

stable for production  ·  canary includes pre-releases  ·  nightly is a weekly snapshot of main

Docker Pulls GitHub Actions Codecov License
Slack Twitter LinkedIn

Introduction

GreptimeDB is an open-source observability database. Metrics, logs, and traces run on one columnar engine over object storage and share one table model: tags, timestamp, and fields. When signals carry common identifiers such as service, host, or trace ID, you can correlate them in SQL without moving data between databases.

Ingest through OpenTelemetry, Prometheus Remote Write, Loki Push, or Elasticsearch Bulk. Use SQL across observability data and PromQL for metrics. Migrate ingestion one signal at a time without rebuilding your collectors.

Why You Might Use It

  • You run Prometheus plus Loki or Elasticsearch and want one backend instead of three
  • You have outgrown Prometheus on cardinality or retention and don't want the Thanos/Mimir operational surface
  • You need long retention on object storage without a separate analytics stack
  • You want to query telemetry with SQL, not only a domain query language
  • You are storing GenAI or agent telemetry (OTel GenAI conventions) alongside infrastructure signals
  • You need the same engine and semantics on resource-constrained devices

Learn more in Why GreptimeDB.

Overview

A quick overview of what GreptimeDB ingests, how it connects to other systems, and what its distributed engine lets you do.

GreptimeDB Overview

What's Supported

Ingest OpenTelemetry (OTLP), Prometheus Remote Write, Loki Push, Elasticsearch Bulk, InfluxDB line protocol, gRPC
Query SQL, PromQL, Jaeger-compatible trace queries, MySQL and PostgreSQL wire protocols
Storage S3, GCS, Azure Blob and S3-compatible endpoints as primary storage, with memory and local-disk caches
Built in Retention policies, downsampling, continuous aggregation, explicit table partitioning, and inverted / skipping / fulltext indexes

Compute and storage are disaggregated: object storage holds the data, while memory and local-disk caches keep recent and frequently queried data close to compute.

Compatibility and Migration

Compatibility is per protocol, and query-side coverage is narrower than ingestion.

Compatible Not compatible
Prometheus Remote Write ingestion; PromQL queries Gaps are listed in PromQL compatibility
Loki Push ingestion; dual-write through Grafana Alloy makes the cutover gradual LogQL and the rest of the Loki query API
Elasticsearch _bulk ingestion in the open-source core; QueryDSL partially, in Enterprise Most other Elasticsearch APIs

Benchmarks:

Limitations and Edition Boundary

Cluster deployment, object storage, the Flow engine, and every ingestion protocol listed above are in the Apache-2.0 build. Repartitioning, region migration, and index creation are manual operations there.

Read replicas, workload isolation, and automated repartitioning are GreptimeDB Enterprise features, along with enterprise security and governance. The Enterprise overview has the current list, and pricing has the edition comparison.

Architecture

GreptimeDB can run in two modes:

  • Standalone — single binary for development and small deployments.
  • Distributed — four components, each independently scalable:
    • Frontend — protocol entry (OTel, Prometheus, MySQL/PostgreSQL, gRPC, ingestion APIs for Elasticsearch/InfluxDB/Loki) and the distributed query engine. Stateless, scales horizontally.
    • Datanode — region engine with WAL, memtable, SST, cache, compaction, and indexes. Persists data to object storage. Elastic.
    • Metasrv — metadata, routing, repartitioning, and security. Backed by a pluggable KV layer (etcd or RDS).
    • Flownode (optional) — continuous flow computation (streaming and materialized views).

For deeper coverage, see the architecture doc or DeepWiki.

GreptimeDB System Overview

Try GreptimeDB

For AI agents — paste this prompt into your agent:

Read https://docs.greptime.com/SKILL.md and follow the instructions
to deploy, configure, ingest, and query GreptimeDB.
docker run -p 127.0.0.1:4000-4003:4000-4003 \
  -v "$(pwd)/greptimedb_data:/greptimedb_data" \
  --name greptime --rm \
  greptime/greptimedb:latest standalone start \
  --http-addr 0.0.0.0:4000 \
  --grpc-bind-addr 0.0.0.0:4001 \
  --mysql-addr 0.0.0.0:4002 \
  --postgres-addr 0.0.0.0:4003

Dashboard: http://localhost:4000/dashboard

Read more in the full Install Guide.

Troubleshooting:

  • Cannot connect to the database? Ensure that ports 4000, 4001, 4002, and 4003 are not blocked by a firewall or used by other services.
  • Failed to start? Check the container logs with docker logs greptime for further details.

Getting Started

Build From Source

Prerequisites:

  • Rust toolchain — nightly, pinned by rust-toolchain.toml
  • Protobuf compiler (>= 3.15)
  • C/C++ building essentials: gcc / g++ / autoconf and the glibc dev package (libc6-dev on Ubuntu, glibc-devel on Fedora)
  • Python toolchain (optional, only for some test scripts)

Build and run:

make                          # build greptime binary
cargo run -- standalone start # start in standalone mode

Common dev commands:

make fmt            # format Rust code
make clippy         # lint (fails on warnings)
make test           # unit + integration tests (uses cargo-nextest)
make sqlness-test   # SQL regression tests

See the Contribution Guidelines for the full developer workflow.

Tools & Extensions

Project Status

GreptimeDB is generally available, with stable APIs and regular releases. It runs in production at scale — OceanBase Cloud operates 80+ GreptimeDB clusters managing 300 TB of logs, cutting log storage cost by 60%+ after migrating from Grafana Loki. See more in case studies.

Read the v1.0 highlights and 2026 roadmap, or browse the version reference.

If GreptimeDB is useful to you, please star the repo.

Known Users

Community

We invite you to engage and contribute!

License

GreptimeDB is an open-core project. Its core is licensed under the Apache License 2.0.

A small set of peripheral, enterprise-only features are gated behind the enterprise Cargo feature (not built by default) and are governed by the separate GreptimeDB Enterprise License. Source files under that license carry an explicit Enterprise License header.

Commercial Support

Scaling observability on your infrastructure? GreptimeDB Enterprise adds the operational, security, and support layer for production deployments. Contact us for details.

Contributing

Acknowledgement

Special thanks to all contributors! See AUTHOR.md.


All trademarks, logos, and brand names referenced in this README and in the Overview diagram are the property of their respective owners. Their use is for identification purposes only and does not imply endorsement or affiliation.

S
Description
Open-source, cloud-native, unified observability database for metrics, logs and traces, supporting SQL/PromQL/Streaming.
Readme Apache-2.0
1.5 GiB
Languages
Rust 97.5%
Python 1.7%
Shell 0.4%
JavaScript 0.2%