Weny Xu 63ef18a74a chore: pick fixes and bump version to v1.1.3 (#8545)
* fix(query): push down PromQL cast filters (#8407)

* fix(query): push down PromQL cast filters

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover cast preimage pushdown

Signed-off-by: discord9 <discord9@163.com>

* test(query): expand cast preimage coverage

Signed-off-by: discord9 <discord9@163.com>

* test(query): compare cast normalization rules

Signed-off-by: discord9 <discord9@163.com>

* test(query): update cast pushdown sqlness plans

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit 4f5dccf6bb)

* fix: update datafusion-pg-catalog (#8417)

(cherry picked from commit 434e0c7682)

* fix: global limit for distributed inspect streams (#8412)

* fix: global limit for distributed inspect streams

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: honor tighter filter fetch under global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: restore distribution after global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: preserve inherited distribution for global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: propagate inherited hash partitioning

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit 5bcc2e9d6b)

* feat: update dashboard to v0.13.7 (#8431)

(cherry picked from commit ebfe4f6eb2)

* fix: preserve repartitioned file refs during GC (#8445)

* fix: preserve repartitioned file refs during GC

Signed-off-by: discord9 <discord9@163.com>

* refactor: clarify file refs target region

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit 149b687a92)

* fix: repartition subset partition key joins (#8460)

Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit ff2fa71d50)

* fix: honor exclusive end in log queries (#8495)

* fix: exclude logs end boundary

Signed-off-by: discord9 <discord9@163.com>

* fix: normalize log query date bounds

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit 99235e4b2d)

* feat: skip oversized compaction tasks (#8466)

fix: skip oversized compaction tasks

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit 0fcf61c3fb)

* feat: use scan output bytes for read costing (#8276)

* feat: track scan output bytes and use them for read costing

Track the actual byte count of each record batch produced by
`RegionScanExec` and use it in place of the aggregated per-plan
`memory_usage` as the `table_scan` cost input. This avoids double
counting bytes that flow through multiple operators.

A named constant `REGION_SCAN_EXEC_NAME` exposes the plan node name
so downstream metric parsers remain correct if the struct is renamed.

Affected files:
- `src/table/src/table/metrics.rs` -- add `output_bytes` counter
- `src/table/src/table/scan.rs` -- record bytes, export name constant
- `src/query/src/dist_plan/merge_scan.rs` -- consume scan bytes

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* refactor: remove misleading `mem_used` gauge from scan metrics

The `mem_used` gauge was used with `add()` on every output batch, but a
RegionScan does not hold memory across polls — each batch is yielded
immediately to the upstream operator. The gauge semantics were
incorrect (cumulative `add()` on a gauge) and the value duplicated
`output_bytes` anyway.

Affected files:
- `src/table/src/table/metrics.rs` — drop `mem_used` field and methods
- `src/table/src/table/scan.rs` — remove `record_mem_usage` call

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix: use stable plan names for scan byte metrics

Expose `plan_name` alongside rendered plan text so scan byte extraction no longer depends on EXPLAIN formatting. Keep old serialized metrics compatible by defaulting missing `plan_name`.

Affected files:
- `src/common/recordbatch/src/adapter.rs` -- add `plan_name` to `PlanMetrics` and populate it from `ExecutionPlan::name`
- `src/query/src/dist_plan/merge_scan.rs` -- match `REGION_SCAN_EXEC_NAME` through `plan_name`

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix: preserve merge scan CPU read cost

Record read cost whenever stream metrics are available, default scan bytes to zero, and aggregate scan output bytes across region scan nodes.

Files: `src/query/src/dist_plan/merge_scan.rs`
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix: correct doc comment in `StreamMetrics`

Fix doc comment on `StreamMetrics::new()` to reference the correct
struct name (`StreamMetrics`) instead of the old `MemoryUsageMetrics`.

- `src/table/src/table/metrics.rs` — fix struct name in doc comment

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

---------

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
(cherry picked from commit 4373c77d35)

* fix: account logical record batch slice memory (#8480)

* fix: account logical record batch slice memory

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: account nested view payload memory

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test: align scan memory expectation with logical size

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: name Arrow inline view limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: account visible list child memory

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: exclude null list child memory

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: exclude null struct child memory

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: account fixed-size list slices

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: limit scan accounting to flat slices

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* docs: clarify nested view accounting scope

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* perf: reduce view memory accounting overhead

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: fix benchmark license header

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* perf: optimize view slice accounting

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test: cover sliced mixed-null view accounting

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: narrow slice accounting to arrow buffers

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* docs: note view accounting assumption

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit 9cde31a647)

* fix: preserve distributed topk merge ordering (#8432)

* fix: preserve distributed topk for scalar latest

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve scalar latest ordering across merge scan

Signed-off-by: discord9 <discord9@163.com>

* fix: carry merge scan ordering from planner rewrite

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan ordering metadata

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan partition ordering gate

Signed-off-by: discord9 <discord9@163.com>

* fix: require adjacent merge sort for ordering metadata

Signed-off-by: discord9 <discord9@163.com>

* test: update ordering sqlness plans

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve distributed topk merge ordering

Signed-off-by: discord9 <discord9@163.com>

* test: cover latest per series queries

Signed-off-by: discord9 <discord9@163.com>

* test: update distributed merge sort sqlness

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan ordering partition gate

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan over-partition ordering

Signed-off-by: discord9 <discord9@163.com>

* test: cover distributed latest with low parallelism

Signed-off-by: discord9 <discord9@163.com>

* fix: keep distributed merge sort opaque

Signed-off-by: discord9 <discord9@163.com>

* test: assert distributed scalar latest merge sort

Signed-off-by: discord9 <discord9@163.com>

* test: clarify merge scan ordering helper

Signed-off-by: discord9 <discord9@163.com>

* fix: complete merge sort exec delegation

Signed-off-by: discord9 <discord9@163.com>

* docs: explain merge sort limit pushdown

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge sort optimizer opacity

Signed-off-by: discord9 <discord9@163.com>

* docs: explain merge sort optimizer hooks

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge sort optimizer hooks

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve merge sort child topk

Signed-off-by: discord9 <discord9@163.com>

* test: update order by topk plan

Signed-off-by: discord9 <discord9@163.com>

* fix: recognize merge sort global fetch

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit b30765e5e9)

* fix: reject out-of-range PostgreSQL numeric UInt64 (#8517)

Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit d48f296144)

* fix(log-store): deduplicate Kafka WAL multipart records (#8514)

* fix(log-store): deduplicate Kafka WAL multipart records

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(log-store): preserve delayed Kafka WAL entry offsets

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(log-store): reject conflicting Kafka WAL last records

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(log-store): emit Kafka WAL entries on last record

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(log-store): discard duplicate Kafka WAL first records

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 912db22417)

* fix: apply log query limit after expressions (#8496)

Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit 1122d1e7d7)

* fix(query): delegate merge sort cardinality effect (#8532)

Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit 887fdf7513)

* fix: upgrade datafusion fork (#8438)

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit f12a1da3de)

* chore: bump version

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
Co-authored-by: sun <sunchang_long@163.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
2026-07-17 11:30:08 +08:00
2023-08-10 08:08:37 +00:00
2023-06-25 11:05:46 +08:00
2023-11-09 10:38:12 +00:00
2023-03-28 19:14:29 +08:00

GreptimeDB Logo

One database for metrics, logs, and traces
replacing Prometheus, Loki, and Elasticsearch

The unified OpenTelemetry backend — with SQL + PromQL on object storage.

Introduction

GreptimeDB is an open-source observability database built for Observability 2.0 — treating metrics, logs, and traces as one unified data model (wide events) instead of three separate pillars.

Use it as the single OpenTelemetry backend — replacing Prometheus, Loki, and Elasticsearch with one database built on object storage. Query with SQL and PromQL, scale without pain, cut costs up to 50×.

Overview

A quick overview of what GreptimeDB ingests, how it connects to other systems, and what its distributed engine lets you do.

GreptimeDB Overview

Features

Feature Description
Observability 2.0 native Logs, metrics, and traces in one engine with SQL + PromQL. Native OpenTelemetry, Prometheus remote write, and Jaeger. Migrate one signal at a time, or use as a single backend.
Elastic compute-storage separation Scale reads independently with horizontal replicas. Serve high-concurrency workloads from dashboards, alerting, and AI agents — without resharding or data migration.
Sub-second on PBEB-scale data Columnar engine with fulltext, inverted, and skipping indexes. Written in Rust. Designed for high-concurrency point queries, not just analytical scans.
50× lower cost Object storage (S3, GCS, Azure Blob) as primary storage, with a tiered cache (memory + local disk) to keep writes and queries fast.

Perfect for:

  • Replacing Prometheus + Loki + Elasticsearch with a single observability backend
  • Scaling past Prometheus — high cardinality, long-term storage, no Thanos/Mimir overhead
  • AI/agent workloads — store GenAI telemetry (OTel GenAI conventions), and serve high-concurrency reads from SRE/developer agents via horizontal read replicas
  • Cutting observability costs with object storage (up to 50× savings on traces, 30% on logs)
  • Edge-to-cloud observability with unified APIs on resource-constrained devices

Why Observability 2.0? Three separate databases for metrics, logs, and traces means three storage layers, three query languages, and three sets of dashboards. GreptimeDB stores all three as timestamped wide events in one columnar engine — JOIN across signals in SQL, run one stack instead of three, and ingest AI agent telemetry the same way. Read more: Observability 2.0 and the Database for It.

Learn more in Why GreptimeDB.

How GreptimeDB Compares

Capability GreptimeDB Prometheus / Thanos / Mimir Grafana Loki Elasticsearch
Data types Metrics, logs, traces Metrics only Logs only Logs, traces
Query language SQL + PromQL PromQL LogQL Query DSL
Storage Native object storage (S3, etc.) Local disk + object storage (Thanos/Mimir) Object storage (chunks) Local disk
Scaling Compute-storage separation, stateless nodes Federation / Thanos / Mimir — multi-component, ops heavy Stateless + object storage Shard-based, ops heavy
Cost efficiency Up to 50× lower storage cost High at scale Moderate High (inverted index overhead)
OpenTelemetry Native (metrics + logs + traces) Partial (metrics only) Partial (logs only) Via instrumentation

Benchmarks:

Architecture

GreptimeDB can run in two modes:

  • Standalone — single binary for development and small deployments.
  • Distributed — four components, each independently scalable:
    • Frontend — protocol entry (OTel, Prometheus, MySQL/PostgreSQL, gRPC, ingestion APIs for Elasticsearch/InfluxDB/Loki) and the distributed query engine. Stateless, scales horizontally.
    • Datanode — region engine with WAL, memtable, SST, cache, compaction, and indexes. Persists data to object storage. Elastic.
    • Metasrv — metadata, routing, repartitioning, autopilot, and security. Backed by a pluggable KV layer (etcd or RDS).
    • Flownode (optional) — continuous flow computation (streaming and materialized views).

For deeper coverage, see the architecture doc or DeepWiki.

GreptimeDB System Overview

Try GreptimeDB

For AI agents — paste this prompt into your agent:

Read https://docs.greptime.com/SKILL.md and follow the instructions
to deploy, configure, ingest, and query GreptimeDB.
docker run -p 127.0.0.1:4000-4003:4000-4003 \
  -v "$(pwd)/greptimedb_data:/greptimedb_data" \
  --name greptime --rm \
  greptime/greptimedb:latest standalone start \
  --http-addr 0.0.0.0:4000 \
  --rpc-bind-addr 0.0.0.0:4001 \
  --mysql-addr 0.0.0.0:4002 \
  --postgres-addr 0.0.0.0:4003

Dashboard: http://localhost:4000/dashboard

Read more in the full Install Guide.

Troubleshooting:

  • Cannot connect to the database? Ensure that ports 4000, 4001, 4002, and 4003 are not blocked by a firewall or used by other services.
  • Failed to start? Check the container logs with docker logs greptime for further details.

Getting Started

Build From Source

Prerequisites:

  • Rust toolchain — nightly, pinned by rust-toolchain.toml
  • Protobuf compiler (>= 3.15)
  • C/C++ building essentials: gcc / g++ / autoconf and the glibc dev package (libc6-dev on Ubuntu, glibc-devel on Fedora)
  • Python toolchain (optional, only for some test scripts)

Build and run:

make                          # build greptime binary
cargo run -- standalone start # start in standalone mode

Common dev commands:

make fmt            # format Rust code
make clippy         # lint (fails on warnings)
make test           # unit + integration tests (uses cargo-nextest)
make sqlness-test   # SQL regression tests

See the Contribution Guidelines for the full developer workflow.

Tools & Extensions

Project Status

GreptimeDB is at v1.0 GA with stable APIs and regular releases. It runs in production at scale — OceanBase Cloud operates 80+ GreptimeDB clusters managing 300 TB of logs, cutting log storage cost by 60% after migrating from Grafana Loki. See more in case studies.

Read the v1.0 highlights and 2026 roadmap, or browse the version reference.

If GreptimeDB is useful to you, please star the repo.

Star History Chart

Known Users

Community

We invite you to engage and contribute!

License

GreptimeDB is licensed under the Apache License 2.0.

Commercial Support

Running GreptimeDB in your organization? We offer enterprise add-ons, services, training, and consulting. Contact us for details.

Contributing

Acknowledgement

Special thanks to all contributors! See AUTHOR.md.


All trademarks, logos, and brand names referenced in this README and in the Overview diagram are the property of their respective owners. Their use is for identification purposes only and does not imply endorsement or affiliation.

S
Description
Languages
Rust 98.8%
Python 0.6%
Shell 0.3%
JavaScript 0.1%