jeremyhi c7fded29ee feat: query mem limiter (#7078)
* feat: query mem limiter

* feat: config docs

* feat: frontend query limit config

* fix: unused imports

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* feat: add metrics for query memory tracker

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: right postion for tracker

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: avoid race condition

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* feat: soft and hard limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* feat: docs

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: when soft_limit == 0

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* feat: upgrade limit algorithm

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: remove batch window

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: batch mem size

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* feat: refine limit algorithm

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: get sys mem

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: minor change

* feat: up tracker to the top stream

* feat: estimated_size for batch

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: minor refactor

* feat: scan_memory_limit connect to max_concurrent_queries

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: make callback clearly

* feat: add unlimted enum

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: by review comment

* chore: comment on recursion_limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* feat: refactor and put permit into RegionScanExec

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* chore: multiple lazy static blocks

* chore: minor change

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2025-11-11 07:47:55 +00:00
2023-08-10 08:08:37 +00:00
2025-11-11 07:47:55 +00:00
2025-06-26 09:18:47 +00:00
2025-11-11 07:47:55 +00:00
2023-06-25 11:05:46 +08:00
2025-11-11 07:47:55 +00:00
2023-11-09 10:38:12 +00:00
2025-11-06 03:21:01 +00:00
2023-03-28 19:14:29 +08:00
2025-04-23 10:48:46 +00:00

GreptimeDB Logo

Real-Time & Cloud-Native Observability Database
for metrics, logs, and traces

Delivers sub-second querying at PB scale and exceptional cost efficiency from edge to cloud.

Introduction

GreptimeDB is an open-source, cloud-native database purpose-built for the unified collection and analysis of observability data (metrics, logs, and traces). Whether youre operating on the edge, in the cloud, or across hybrid environments, GreptimeDB empowers real-time insights at massive scale — all in one system.

Features

Feature Description
Unified Observability Data Store metrics, logs, and traces as timestamped, contextual wide events. Query via SQL, PromQL, and streaming.
High Performance & Cost Effective Written in Rust, with a distributed query engine, rich indexing, and optimized columnar storage, delivering sub-second responses at PB scale.
Cloud-Native Architecture Designed for Kubernetes, with compute/storage separation, native object storage (AWS S3, Azure Blob, etc.) and seamless cross-cloud access.
Developer-Friendly Access via SQL/PromQL interfaces, REST API, MySQL/PostgreSQL protocols, and popular ingestion protocols.
Flexible Deployment Deploy anywhere: edge (including ARM/Android) or cloud, with unified APIs and efficient data sync.

Learn more in Why GreptimeDB and Observability 2.0 and the Database for It.

Quick Comparison

Feature GreptimeDB Traditional TSDB Log Stores
Data Types Metrics, Logs, Traces Metrics only Logs only
Query Language SQL, PromQL, Streaming Custom/PromQL Custom/DSL
Deployment Edge + Cloud Cloud/On-prem Mostly central
Indexing & Performance PB-Scale, Sub-second Varies Varies
Integration REST, SQL, Common protocols Varies Varies

Performance:

Read more benchmark reports.

Architecture

Try GreptimeDB

docker pull greptime/greptimedb
docker run -p 127.0.0.1:4000-4003:4000-4003 \
  -v "$(pwd)/greptimedb_data:/greptimedb_data" \
  --name greptime --rm \
  greptime/greptimedb:latest standalone start \
  --http-addr 0.0.0.0:4000 \
  --rpc-bind-addr 0.0.0.0:4001 \
  --mysql-addr 0.0.0.0:4002 \
  --postgres-addr 0.0.0.0:4003

Dashboard: http://localhost:4000/dashboard Full Install Guide

Troubleshooting:

  • Cannot connect to the database? Ensure that ports 4000, 4001, 4002, and 4003 are not blocked by a firewall or used by other services.
  • Failed to start? Check the container logs with docker logs greptime for further details.

Getting Started

Build From Source

Prerequisites:

  • Rust toolchain (nightly)
  • Protobuf compiler (>= 3.15)
  • C/C++ building essentials, including gcc/g++/autoconf and glibc library (eg. libc6-dev on Ubuntu and glibc-devel on Fedora)
  • Python toolchain (optional): Required only if using some test scripts.

Build and Run:

make
cargo run -- standalone start

Tools & Extensions

Project Status

Status: Beta. GA (v1.0): Targeted for mid 2025.

  • Being used in production by early adopters
  • Stable, actively maintained, with regular releases (version info)
  • Suitable for evaluation and pilot deployments

For production use, we recommend using the latest stable release. Star History Chart

If you find this project useful, a would mean a lot to us! Known Users

Community

We invite you to engage and contribute!

License

GreptimeDB is licensed under the Apache License 2.0.

Commercial Support

Running GreptimeDB in your organization? We offer enterprise add-ons, services, training, and consulting. Contact us for details.

Contributing

Acknowledgement

Special thanks to all contributors! See AUTHORS.md.

Description
Languages
Rust 99.6%