discord9
b3c756a7dd
* perf(promql): zero-copy timestamp payload for millisecond zero-offset ranges RangeManipulate rebuilt the timestamp range payload per sample through wide-precision i128 conversion even for millisecond input with zero offset, where the input is already millisecond and nothing changes. That turned a previously zero-copy path into a per-sample loop plus a Vec allocation, regressing range-function queries. Reuse the input timestamp array directly when the time index is already millisecond and the offset is zero. Other units and non-zero offsets keep the wide-precision conversion (required so ns/us data near the native type limits stays correct under large offsets). Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Signed-off-by: discord9 <discord9@163.com> * perf(promql): vectorize timestamp payload with plain i64 RangeManipulate rebuilt the timestamp range payload per sample through wide-precision i128 conversion, turning a hot path into a scalar loop with a Vec allocation. Realistic timestamps and offsets stay far inside i64, so the wide path is unnecessary. - millisecond + zero offset reuses the input array with zero copy; - other units / offsets shift in one vectorized i64 pass (no i128, no per-sample Option, no overflow branch). Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Signed-off-by: discord9 <discord9@163.com> * perf(promql): check timestamp payload overflow via arrow try_unary Replace the silent wrapping shift with try_unary + checked_mul/checked_add. checked arithmetic lowers to a per-lane overflow flag so the single pass still vectorizes over the dense i64 buffer; any overflow short-circuits to an error instead of silently wrapping to a wrong timestamp. No i128, no per-sample Option. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Signed-off-by: discord9 <discord9@163.com> * perf(promql): vectorize cursor scans with i64 fast path RangeManipulate::calculate_range and InstantManipulate::manipulate compared every sample through i128 wide-precision arithmetic. For realistic timestamps and offsets the shifted values stay far inside i64, so the wide path was pure overhead. Split both scans into an i64 fast path (checked arithmetic precomputed once per batch) and an i128 exact fallback. Any input that would overflow i64 falls back to the i128 scan, keeping window membership semantics identical. Verified: 47/47 range_manipulate + instant_manipulate tests, 275/275 promql crate tests, 1612 randomized cases byte-identical to i128 scan. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Signed-off-by: discord9 <discord9@163.com> * refactor(promql): generalize cursor scans and share endpoint precheck Address review comments on PR #9417: - Generalize scan_ranges/scan_instants over Nanoseconds (i64/i128), eliminating the duplicated scan bodies. - Extract shared narrow_scan precheck used by both RangeManipulate and InstantManipulate, with identical checked endpoint logic. - Trim doc comments to one line; fix factual errors about fallback trigger (2262 for ms/s, not native type limits). Semantics unchanged: same cursor algorithm, same overflow fallback, same i64-vs-i128 selection, same stale/null/tie handling. Verified: 47/47 range_manipulate + instant_manipulate tests, clippy clean. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Signed-off-by: discord9 <discord9@163.com> * perf(promql): vectorize payload with try_unary and add scan carrier bench Replace the per-sample i128 payload loop with a single arrow try_unary pass: i128 intermediate holds any native timestamp shift, only the final millisecond result must fit i64. This is semantically equivalent to the original i128 implementation (no silent wrap, no per-sample branching). Add cursor_scan_carriers A/B benchmark: i64 fast path vs i128 exact fallback on four workloads. Add five deterministic differential tests pinning scan_ranges::<i64> and scan_ranges::<i128> to be pointwise equivalent on boundary shapes (sub-ms samples, fallback triggers, nanosecond extremes, seeded precision matrix). Bench (N=4096, 15s cadence): - payload wall_time 20x15s: 74.94us -> 48.98us (-35%) - payload wall_time 240x15s: 102.44us - scan i64 fast path: 10.97us vs i128 19.84us (1.81x) Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * perf(promql): restore zero-copy payload for ms+offset=0 and fix comment Restore the zero-copy fast path for millisecond timestamps with zero offset, guarded by the input column's exact data type (Timestamp (Millisecond, None)) rather than self.time_unit. The old time_unit guard would accept a tz-carrying millisecond column and produce a dictionary value type disagreeing with the output schema; the datatype guard is the correct check. Also fix the comment: arrow's try_unary is a fallible kernel that LLVM cannot auto-vectorize; the win over the hand-written loop comes from eliminating the per-sample Option/Vec and branches, not SIMD. Add a regression test with a timezone-carrying millisecond index to prove it falls back to the try_unary path. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * perf(promql): replace try_unary with unary + Cell overflow flag Arrow's try_unary is a fallible kernel that LLVM cannot auto-vectorize. Replace it with the infallible unary kernel: the closure is a plain i64 -> i64 mapping that sets a Cell<bool> on overflow instead of returning Err per sample. The flag is checked once after the loop. This removes the per-lane Result construction and lets LLVM vectorize the shift+truncate sequence. Bench (N=4096, 15s cadence): - window4x15s: 42.12us -> 28.77us (-32%) - window20x15s: 48.98us -> 27.68us (-43%) - window240x15s: 102.44us -> 27.61us (-73%) - regular_sparse: 90.04us -> 29.20us (-68%) All cases now run at ~27-29us regardless of window size; the zero-copy path dominates for ms+offset=0. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * perf(promql): dispatch payload conversion on null_count; drop wall-time bench Null-free batches (dominant PromQL case) take arrow's vectorizable unary kernel with a Cell<bool> overflow flag; nullable batches take try_unary so a NULL slot's arbitrary underlying value cannot raise a false batch-wide overflow error. Add a regression test exercising both paths. Remove bench_range_manipulate_wall_time (all-ms zero-offset cases only exercised the zero-copy path, measuring nothing about the conversion) and its orphaned helpers; restore the cursor-scan bench helpers it shared. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> * fix(promql): keep nullable timestamp cursor scans on i128 NULL slots may contain unordered arbitrary values, so the endpoint-only narrow-scan proof cannot bound their interior timestamp arithmetic. Both range and instant scans now attempt i64 only for null-free timestamp arrays and otherwise retain the existing exact i128 path. Add focused interior-NULL regressions through both active entrypoints, checking concrete selections and output against the existing wide scan. The null-free fast path and timestamp payload conversion are unchanged. Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> --------- Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com> Signed-off-by: discord9 <discord9@163.com>
Metrics, logs, and traces.
One engine, on your infrastructure.
A columnar database for metrics, logs, and traces on object storage. Apache-2.0 licensed core.
User Guide · API Docs · Roadmap 2026 · Slack
stable for production · canary includes pre-releases · nightly is a weekly snapshot of main
Introduction
GreptimeDB is an open-source observability database. Metrics, logs, and traces run on one columnar engine over object storage and share one table model: tags, timestamp, and fields. When signals carry common identifiers such as service, host, or trace ID, you can correlate them in SQL without moving data between databases.
Ingest through OpenTelemetry, Prometheus Remote Write, Loki Push, or Elasticsearch Bulk. Use SQL across observability data and PromQL for metrics. Migrate ingestion one signal at a time without rebuilding your collectors.
One Query Across Signals
OpenTelemetry ingestion writes spans to opentelemetry_traces and log records to
opentelemetry_logs. Both tables carry trace_id, so correlating them is a join:
-- The slowest failed spans in the last hour,
-- with the log lines emitted inside those same traces.
SELECT
t.service_name,
t.span_name,
t.duration_nano / 1000000 AS duration_ms,
l.timestamp AS log_time,
l.severity_text,
l.body
FROM opentelemetry_traces t
JOIN opentelemetry_logs l ON l.trace_id = t.trace_id
WHERE t.timestamp > now() - INTERVAL '1' HOUR
AND t.span_status_code = 'STATUS_CODE_ERROR'
ORDER BY t.duration_nano DESC
LIMIT 20;
Metrics join the same way, on any tag the tables share, such as service,
host, or pod.
Why You Might Use It
- You run Prometheus plus Loki or Elasticsearch and want one backend instead of three
- You have outgrown Prometheus on cardinality or retention and don't want the Thanos/Mimir operational surface
- You are hitting Loki's query performance limits as log volume grows
- You need long retention on object storage without a separate analytics stack
- You want to query telemetry with SQL, not only a domain query language
- You are storing GenAI or agent telemetry (OTel GenAI conventions) alongside infrastructure signals
Learn more in Why GreptimeDB.
What's Supported
| Ingest | OpenTelemetry (OTLP), Prometheus Remote Write, Loki Push, Elasticsearch Bulk, InfluxDB line protocol, gRPC |
| Query | SQL, PromQL, Jaeger-compatible trace queries, MySQL and PostgreSQL wire protocols |
| Storage | S3, GCS, Azure Blob and S3-compatible endpoints as primary storage, with memory and local-disk caches |
| Built in | Retention policies, downsampling, continuous aggregation, explicit table partitioning, and inverted / skipping / fulltext indexes |
Compute and storage are disaggregated: object storage holds the data, while memory and local-disk caches keep recent and frequently queried data close to compute.
Benchmarks
- Agent RCA Bench: LLM agents doing root cause analysis over GreptimeDB versus Prometheus + Loki + Tempo. 40% fewer wrong diagnoses, 48% fewer input tokens, 45% lower cost (write-up)
- GreptimeDB tops JSONBench's billion-record cold run test
- TSBS Benchmark
- More benchmark reports
Compatibility and Migration
Compatibility is per protocol, and query-side coverage is narrower than ingestion.
| Compatible | Not compatible | |
|---|---|---|
| Prometheus | Remote Write ingestion; PromQL queries | Gaps are listed in PromQL compatibility |
| Loki | Push ingestion; dual-write through Grafana Alloy makes the cutover gradual | LogQL and the rest of the Loki query API |
| Elasticsearch | _bulk ingestion in the open-source core; QueryDSL partially, in Enterprise |
Most other Elasticsearch APIs |
Limitations and Edition Boundary
Cluster deployment, object storage, the Flow engine, and every ingestion protocol listed above are in the Apache-2.0 build. Repartitioning, region migration, and index creation are manual operations there.
Read replicas, workload isolation, and automated repartitioning are GreptimeDB Enterprise features, along with enterprise security and governance. The Enterprise overview has the current list, and pricing has the edition comparison.
Architecture
GreptimeDB can run in two modes:
- Standalone — single binary for development and small deployments.
- Distributed — four components, each independently scalable:
- Frontend — protocol entry (OTel, Prometheus, MySQL/PostgreSQL, gRPC, ingestion APIs for Elasticsearch/InfluxDB/Loki) and the distributed query engine. Stateless, scales horizontally.
- Datanode — region engine with WAL, memtable, SST, cache, compaction, and indexes. Persists data to object storage. Elastic.
- Metasrv — metadata, routing, repartitioning, and security. Backed by a pluggable KV layer (etcd or RDS).
- Flownode (optional) — continuous flow computation (streaming and materialized views).
For deeper coverage, see the architecture doc or DeepWiki.
Try GreptimeDB
For AI agents — paste this prompt into your agent:
Read https://docs.greptime.com/SKILL.md and follow the instructions
to deploy, configure, ingest, and query GreptimeDB.
docker run -p 127.0.0.1:4000-4003:4000-4003 \
-v "$(pwd)/greptimedb_data:/greptimedb_data" \
--name greptime --rm \
greptime/greptimedb:latest standalone start \
--http-addr 0.0.0.0:4000 \
--grpc-bind-addr 0.0.0.0:4001 \
--mysql-addr 0.0.0.0:4002 \
--postgres-addr 0.0.0.0:4003
Dashboard: http://localhost:4000/dashboard
Read more in the full Install Guide.
Troubleshooting:
- Cannot connect to the database? Ensure that ports
4000,4001,4002, and4003are not blocked by a firewall or used by other services. - Failed to start? Check the container logs with
docker logs greptimefor further details.
Getting Started
Build From Source
Prerequisites:
- Rust toolchain — stable, pinned by
rust-toolchain.toml - Protobuf compiler (>= 3.15)
- C/C++ building essentials:
gcc/g++/autoconfand the glibc dev package (libc6-devon Ubuntu,glibc-develon Fedora) - Python toolchain (optional, only for some test scripts)
Build and run:
make # build greptime binary
cargo run -- standalone start # start in standalone mode
Common dev commands:
make fmt # format Rust code
make clippy # lint (fails on warnings)
make test # unit + integration tests (uses cargo-nextest)
make sqlness-test # SQL regression tests
See the Contribution Guidelines for the full developer workflow.
Tools & Extensions
- Kubernetes: GreptimeDB Operator
- Helm Charts: Greptime Helm Charts
- Dashboard: Web UI
- gRPC Ingester: Go, Java, C++, Erlang, Rust, .NET, TypeScript
- Grafana Data Source: GreptimeDB Grafana data source plugin
- Grafana Dashboard: Official Dashboard for monitoring
Project Status
GreptimeDB is generally available, with stable APIs and regular releases. It runs in production at scale — OceanBase Cloud operates 80+ GreptimeDB clusters managing 300 TB of logs, cutting log storage cost by 60%+ after migrating from Grafana Loki. See more in case studies.
Release lines and support windows are in the version reference. For where the project is going, read the v1.0 highlights and the 2026 roadmap.
Community
We invite you to engage and contribute!
If GreptimeDB is useful to you, please star the repo.
License
GreptimeDB is an open-core project. Its core is licensed under the Apache License 2.0.
A small set of peripheral, enterprise-only features are gated behind the
enterprise Cargo feature (not built by default) and are governed by the
separate GreptimeDB Enterprise License. Source files under
that license carry an explicit Enterprise License header.
Commercial Support
Scaling observability on your infrastructure? GreptimeDB Enterprise adds the operational, security, and support layer for production deployments. Contact us for details.
Contributing
- Read our Contribution Guidelines.
- Explore Internal Concepts and DeepWiki.
- Pick up a good first issue and join the #contributors Slack channel.
Acknowledgement
Special thanks to all contributors! See AUTHOR.md.
- Uses Apache Arrow™ (memory model)
- Apache Parquet™ (file storage)
- Apache DataFusion™ (query engine)
- Apache OpenDAL™ (data access abstraction)
All trademarks, logos, and brand names referenced in this README and in the Overview diagram are the property of their respective owners. Their use is for identification purposes only and does not imply endorsement or affiliation.