jeremyhi
50a2c4d378
* fix(mito2): publish manifest deltas by conditional create under a lease
A late old leader and the region's new holder can commit the same manifest
version. For a region whose writes require a lease and whose WAL is the object
store WAL, the manifest now behaves as follows. Every other region keeps its
behaviour and issues no extra request.
- Each delta has one fixed name, `{version:020}.json.gz`, written by a
conditional create. Without `compress_manifest` the delta is written at gzip
level 0, so that all writers share the name.
- On an existing object, the writer reads it from storage: identical content is
a retry of a write that landed, anything else is a retryable
`ManifestVersionConflict` that keeps the output files.
- The version advances only after a successful save.
- Recovery loads the highest checkpoint found by LIST and uses the checkpoint
pointer only for its checksum, because a stale writer's checkpoint task can
leave the pointer at an older version.
- Checkpoint cleanup frees names, so a late create can still succeed on a
deleted name. A commit whose save returns after the region's Leader deadline
lists the checkpoints once: a checkpoint at or above its version, or a LIST
error, is a conflict; otherwise it applies but starts no checkpoint. A commit
within the deadline does one conditional PUT and no LIST.
`RegionManifestManager::update` becomes the one commit entry and takes
`allow_checkpoint` and `must_verify` as required parameters. A backend that does
not declare conditional create is refused at open.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(mito2): treat every LIST error as a conflict when verifying a commit
The listing helper turned a NotFound from lister creation into an empty result,
so the check after a late commit could read a failed LIST as proof that no
checkpoint covers the version. List the checkpoints directly, so that any LIST
error reaches the caller. Recovery still treats a missing directory as having no
checkpoint.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(mito2): keep the LIST failure in manifest conflicts and follow the style guide
A failed LIST during commit verification was reported as "OpenDAL operator
failed", because the conflict reason used the display of the wrapper error.
Report the root cause through `output_msg` and assert it in the test.
Align the change with docs/style-guide.md: name boolean methods `is_*`, name
the listing helper `list_highest_checkpoint` and move it after the public
methods, qualify `encode_gzip_stored` through its module, and name the mode
setter after the field it sets, which is now `pub(crate)`. Move the shared test
helpers to `crate::test_util` and `manifest/tests/utils.rs` so that the manifest
test modules stay private.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(mito2): order the manifest test helpers public before private
Move the private `NotFoundReader` and `CheckpointTaskBlocker::block_write`
below the public items next to them, and document
`fail_manifest_delta_reads_layer`.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(mito2): fail recovery when a listed checkpoint disappears
Recovery treated a checkpoint that LIST found but that cleanup removed before
the read as no checkpoint at all. It then replayed from the first version over
deltas that were already deleted, and could report an empty region or build a
manifest from the surviving tail. Return a retryable version conflict instead.
Record the size of the stored object when a retry finds an identical delta, as
the other writer may have encoded it differently. Reopen the region in the test
of a commit that crosses its deadline, and reduce the AGENTS.md entry to the
invariant.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(mito2): verify conditional commits from callers without a deadline
Restore `RegionManifestManager::update(action_list, is_staging)`. Without a
Leader deadline a caller cannot show that its commit returned in time, so a
normal commit under conditional publication is now always verified on this
path and starts no checkpoint.
Only `ManifestContext::update_locked` has a deadline, so the variant that takes
`allow_checkpoint` and `must_verify` becomes `update_with_verification` and is
crate private. `sst-replace` and the existing tests return to the two-argument
call.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(mito2): name the verifying manifest update in the hook docs
`ManifestContext::update_locked` now calls `update_with_verification`, and the
public `update` also calls it, so neither doc can call `update_locked` the sole
caller of `RegionManifestManager::update`.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(mito2): keep the conditional publication rule in region options
Move the rule that decides which regions publish manifests conditionally
to RegionOptions, so the manifest options only hold the result. Report a
checkpoint that vanished after LIST with its own error instead of a
version conflict, and keep the stored gzip encoder with the other
CompressionType helpers.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Metrics, logs, and traces.
One engine, on your infrastructure.
User Guide · API Docs · Live Demo · Roadmap 2026 · Slack
stable for production · canary includes pre-releases · nightly is a weekly snapshot of main
Introduction
GreptimeDB is an open-source observability database. Metrics, logs, and traces run on one columnar engine over object storage and share one table model: tags, timestamp, and fields. When signals carry common identifiers such as service, host, or trace ID, you can correlate them in SQL without moving data between databases.
Ingest through OpenTelemetry, Prometheus Remote Write, Loki Push, or Elasticsearch Bulk. Use SQL across observability data and PromQL for metrics. Migrate ingestion one signal at a time without rebuilding your collectors.
One Query Across Signals
OpenTelemetry ingestion writes spans to opentelemetry_traces and log records to
opentelemetry_logs. Both tables carry trace_id, so correlating them is a join:
-- The slowest failed spans in the last hour,
-- with the log lines emitted inside those same traces.
SELECT
t.service_name,
t.span_name,
t.duration_nano / 1000000 AS duration_ms,
l.timestamp AS log_time,
l.severity_text,
l.body
FROM opentelemetry_traces t
JOIN opentelemetry_logs l ON l.trace_id = t.trace_id
WHERE t.timestamp > now() - INTERVAL '1' HOUR
AND t.span_status_code = 'STATUS_CODE_ERROR'
ORDER BY t.duration_nano DESC
LIMIT 20;
Metrics join the same way, on any tag the tables share, such as service,
host, or pod.
Try GreptimeDB
For AI agents — paste this prompt into your agent:
Read https://docs.greptime.com/SKILL.md and follow the instructions
to deploy, configure, ingest, and query GreptimeDB.
docker run -p 127.0.0.1:4000-4003:4000-4003 \
-v "$(pwd)/greptimedb_data:/greptimedb_data" \
--name greptime --rm \
greptime/greptimedb:latest standalone start \
--http-addr 0.0.0.0:4000 \
--grpc-bind-addr 0.0.0.0:4001 \
--mysql-addr 0.0.0.0:4002 \
--postgres-addr 0.0.0.0:4003
Dashboard: http://localhost:4000/dashboard
Or skip the install and try the Live Demo.
Next: Quickstart, the full Install Guide, Demo Scenes, and the FAQ.
Why You Might Use It
- You run Prometheus plus Loki or Elasticsearch and want one backend instead of three
- You have outgrown Prometheus on cardinality or retention and don't want the Thanos/Mimir operational surface
- You are hitting Loki's query performance limits as log volume grows
- You need long retention on object storage without a separate analytics stack
- You want to query telemetry with SQL, not only a domain query language
- You are storing GenAI or agent telemetry (OTel GenAI conventions) alongside infrastructure signals
Learn more in Why GreptimeDB.
What's Supported
| Ingest | OpenTelemetry (OTLP), Prometheus Remote Write (with native histograms over Remote Write 2.0), Loki Push, Elasticsearch Bulk, InfluxDB line protocol, gRPC |
| Query | SQL, PromQL, Jaeger-compatible trace queries, MySQL and PostgreSQL wire protocols |
| Storage | S3, GCS, Azure Blob and S3-compatible endpoints as primary storage, with memory and local-disk caches |
| Built in | Pipelines that parse and transform logs at ingestion, retention policies, downsampling, continuous aggregation, explicit table partitioning, and inverted / skipping / fulltext indexes |
Compute and storage are disaggregated: object storage holds the data, while memory and local-disk caches keep recent and frequently queried data close to compute.
Benchmarks
- Agent RCA Bench: LLM agents doing root cause analysis over GreptimeDB versus Prometheus + Loki + Tempo, 168 investigations per interface across six models. 40% fewer wrong diagnoses, about 48% fewer input tokens, about 45% lower estimated cost (write-up)
- JSONBench billion-record cold run: first place as of March 2025
- TSBS Benchmark
- More benchmark reports
Compatibility and Migration
Compatibility is per protocol, and query-side coverage is narrower than ingestion.
| Compatible | Not compatible | |
|---|---|---|
| Prometheus | Remote Write ingestion; PromQL queries | Gaps are listed in PromQL compatibility |
| Loki | Push ingestion; dual-write through Grafana Alloy makes the cutover gradual | LogQL and the rest of the Loki query API |
| Elasticsearch | _bulk ingestion in the open-source core; QueryDSL partially, in Enterprise |
Most other Elasticsearch APIs |
Limitations and Edition Boundary
Cluster deployment, object storage, the Flow engine, and every ingestion protocol listed above are in the Apache-2.0 build. Repartitioning and region migration are manual operations there.
Read replicas, workload isolation, and automated repartitioning are GreptimeDB Enterprise features, along with enterprise security and governance. The Enterprise overview has the current list, and pricing has the edition comparison. For GreptimeDB Enterprise inquiries, contact us.
Architecture
GreptimeDB can run in two modes:
- Standalone — single binary for development and small deployments.
- Distributed — four components, each independently scalable:
- Frontend — protocol endpoints and the distributed query engine. Stateless, scales horizontally.
- Datanode — region engine with WAL, memtable, SST, cache, compaction, and indexes. Persists data to object storage.
- Metasrv — metadata, region routing, and cluster coordination (DDL procedures, failover, region migration, repartitioning). Backed by a pluggable KV layer (etcd or RDS).
- Flownode (optional) — continuous flow computation (streaming and materialized views).
For deeper coverage, see the architecture doc or DeepWiki.
Build From Source
Prerequisites:
- Rust toolchain — stable, pinned by
rust-toolchain.toml - Protobuf compiler (>= 3.15)
- C/C++ building essentials:
gcc/g++/autoconfand the glibc dev package (libc6-devon Ubuntu,glibc-develon Fedora) - Python toolchain (optional, only for some test scripts)
Build and run:
make # build greptime binary
cargo run -- standalone start # start in standalone mode
Common dev commands:
make fmt # format Rust code
make clippy # lint (fails on warnings)
make test # unit + integration tests (uses cargo-nextest)
make sqlness-test # SQL regression tests
See the Contribution Guidelines for the full developer workflow.
Tools & Extensions
- Kubernetes: GreptimeDB Operator
- Helm Charts: Greptime Helm Charts
- Dashboard: Web UI
- gRPC Ingester: Go, Java, C++, Erlang, Rust, .NET, TypeScript
- Grafana Data Source: GreptimeDB Grafana data source plugin
- Grafana Dashboard: Official Dashboard for monitoring
Project Status
GreptimeDB is generally available, with stable APIs and regular releases. It runs in production at scale — OceanBase Cloud operates 80+ GreptimeDB clusters managing 300 TB of logs, cutting log storage cost by 60%+ after migrating from Grafana Loki. See more in case studies.
Releases follow Semantic Versioning. For where the project is going, read the v1.0 highlights, the 2026 roadmap, and what we mean by an observability database.
Community
If GreptimeDB is useful to you, please star the repo.
License
GreptimeDB is an open-core project. Its core is licensed under the Apache License 2.0.
A small set of peripheral, enterprise-only features are gated behind the
enterprise Cargo feature (not built by default) and are governed by the
separate GreptimeDB Enterprise License. Source files under
that license carry an explicit Enterprise License header.
Contributing
- Read our Contribution Guidelines.
- Explore Internal Concepts and DeepWiki.
- Pick up a good first issue and join the #contributors Slack channel.
Acknowledgement
Special thanks to all contributors! See AUTHOR.md.
- Uses Apache Arrow™ (memory model)
- Apache Parquet™ (file storage)
- Apache DataFusion™ (query engine)
- Apache OpenDAL™ (data access abstraction)
All trademarks, logos, and brand names referenced in this README and in the Overview diagram are the property of their respective owners. Their use is for identification purposes only and does not imply endorsement or affiliation.