40b52f3b13
* chore: add Noop Wal option * remove: WalOptionsAllocator::alloc method * feat/no-op-wal: ### Add Noop WAL Option - **`engine.rs`, `opener.rs`, `wal.rs`, `entry_reader.rs`, `handle_write.rs`, `provider.rs`**: - Introduced a new `WalOptions::Noop` variant to handle scenarios where no write-ahead logging is required. - Implemented `NoopEntryReader` to provide a no-operation entry reader. - Updated logic to skip WAL operations for regions with `Noop` option. - Added `Provider::Noop` to handle `Noop` operations in the provider logic. * feat/no-op-wal: ### Add `skip_wal` Option to Table Metadata - **Enhancements in `table_meta.rs`**: - Added a `skip_wal` parameter to the `create_wal_options` function to allow skipping WAL writes. - Updated the `create_table_route` function to utilize the `skip_wal` option from `table_info.meta.options`. - **Updates in `wal_options_allocator.rs`**: - Modified `alloc_batch` to handle the `skip_wal` flag, setting WAL options to `Noop` when true. - Added a test case `test_allocator_with_skip_wal` to verify the `skip_wal` functionality. - **Changes in `requests.rs`**: - Introduced `skip_wal` in `TableOptions` and added parsing logic. - Updated `TableOptions` display to include `skip_wal`. These changes introduce the ability to skip WAL writes for tables, enhancing flexibility in table metadata management. * feat/no-op-wal: **Add WAL Option Handling and Table Option Validation** - **`handle_write.rs`**: Introduced a check for `WalOptions::Noop` in the `RegionWorkerLoop` to skip WAL writing for regions with this option. - **`requests.rs`**: Added `SKIP_WAL_KEY` to the list of valid table options for enhanced table configuration validation. * feat/no-op-wal: ### Update WAL Options Allocation - **`key.rs`**: Modified the `allocate_region_wal_options` function to include an additional boolean parameter, enhancing the allocation logic. - **`wal_options_allocator.rs`**: Simplified the `test_allocator_with_skip_wal` test by removing unnecessary variable declarations and directly using `WalOptionsAllocator::RaftEngine`. These changes improve the flexibility and efficiency of WAL options allocation in the system. * chore: reformat code * feat/no-op-wal: **Enhancement:** Conditional Addition of `SKIP_WAL_KEY` in `requests.rs` - Updated `TableOptions` implementation in `requests.rs` to conditionally add `SKIP_WAL_KEY` to `key_vals` only when `self.skip_wal` is true, optimizing the key-value pair generation. * feat/no-op-wal: Update `requests.rs` tests to reflect changes in `skip_wal` option - Modified test assertions in `requests.rs` to remove `skip_wal=false` from expected strings. - Added a new test case to verify `skip_wal=true` is correctly represented in `TableOptions`. * feat/no-op-wal: Add Debug Logging and Improve Error Handling for WAL and Table Options • Introduced debug logging in wal.rs to skip obsolete regions, enhancing traceability. • Improved error handling in requests.rs by replacing warn with error propagation for invalid skip_wal values. • Added new test cases for skip_wal functionality, including SQL scripts and expected results, to ensure correct behavior and validation of the changes.
Unified & Cost-Effective Observerability Database for Metrics, Logs, and Events
- Introduction
- Features: Why GreptimeDB
- Architecture
- Try it for free
- Getting Started
- Project Status
- Join the community
- Tools & Extensions
- License
- Acknowledgement
Introduction
GreptimeDB is an open-source unified & cost-effective observerability database for Metrics, Logs, and Events (also Traces in plan). You can gain real-time insights from Edge to Cloud at Any Scale.
News
GreptimeDB archives 1 billion cold run #1 in JSONBench!
Why GreptimeDB
Our core developers have been building observerability data platforms for years. Based on our best practices, GreptimeDB was born to give you:
-
Unified Processing of Metrics, Logs, and Events
GreptimeDB unifies observerability data processing by treating all data - whether metrics, logs, or events - as timestamped events with context. Users can analyze this data using either SQL or PromQL and leverage stream processing (Flow) to enable continuous aggregation. Read more.
-
Cloud-native Distributed Database
Built for Kubernetes. GreptimeDB achieves seamless scalability with its cloud-native architecture of separated compute and storage, built on object storage (AWS S3, Azure Blob Storage, etc.) while enabling cross-cloud deployment through a unified data access layer.
-
Performance and Cost-effective
Written in pure Rust for superior performance and reliability. GreptimeDB features a distributed query engine with intelligent indexing to handle high cardinality data efficiently. Its optimized columnar storage achieves 50x cost efficiency on cloud object storage through advanced compression. Benchmark reports.
-
Cloud-Edge Collaboration
GreptimeDB seamlessly operates across cloud and edge (ARM/Android/Linux), providing consistent APIs and control plane for unified data management and efficient synchronization. Learn how to run on Android.
-
Multi-protocol Ingestion, SQL & PromQL Ready
Widely adopted database protocols and APIs, including MySQL, PostgreSQL, InfluxDB, OpenTelemetry, Loki and Prometheus, etc. Effortless Adoption & Seamless Migration. Supported Protocols Overview.
For more detailed info please read Why GreptimeDB.
Try GreptimeDB
1. Live Demo
Try out the features of GreptimeDB right from your browser.
2. GreptimeCloud
Start instantly with a free cluster.
3. Docker Image
To install GreptimeDB locally, the recommended way is via Docker:
docker pull greptime/greptimedb
Start a GreptimeDB container with:
docker run -p 127.0.0.1:4000-4003:4000-4003 \
-v "$(pwd)/greptimedb:./greptimedb_data" \
--name greptime --rm \
greptime/greptimedb:latest standalone start \
--http-addr 0.0.0.0:4000 \
--rpc-bind-addr 0.0.0.0:4001 \
--mysql-addr 0.0.0.0:4002 \
--postgres-addr 0.0.0.0:4003
Access the dashboard via http://localhost:4000/dashboard.
Read more about Installation on docs.
Getting Started
Build
Check the prerequisite:
- Rust toolchain (nightly)
- Protobuf compiler (>= 3.15)
- C/C++ building essentials, including
gcc/g++/autoconfand glibc library (eg.libc6-devon Ubuntu andglibc-develon Fedora) - Python toolchain (optional): Required only if using some test scripts.
Build GreptimeDB binary:
make
Run a standalone server:
cargo run -- standalone start
Tools & Extensions
Kubernetes
Dashboard
SDK
- GreptimeDB Go Ingester
- GreptimeDB Java Ingester
- GreptimeDB C++ Ingester
- GreptimeDB Erlang Ingester
- GreptimeDB Rust Ingester
- GreptimeDB JavaScript Ingester
Grafana Dashboard
Our official Grafana dashboard for monitoring GreptimeDB is available at grafana directory.
Project Status
GreptimeDB is currently in Beta. We are targeting GA (General Availability) with v1.0 release by Early 2025.
While in Beta, GreptimeDB is already:
- Being used in production by early adopters
- Actively maintained with regular releases, about version number
- Suitable for testing and evaluation
For production use, we recommend using the latest stable release.
Community
Our core team is thrilled to see you participate in any ways you like. When you are stuck, try to ask for help by filling an issue with a detailed description of what you were trying to do and what went wrong. If you have any questions or if you would like to get involved in our community, please check out:
- GreptimeDB Community on Slack
- GreptimeDB GitHub Discussions forum
- Greptime official website
In addition, you may:
Commercial Support
If you are running GreptimeDB OSS in your organization, we offer additional enterprise add-ons, installation services, training, and consulting. Contact us and we will reach out to you with more detail of our commercial license.
License
GreptimeDB uses the Apache License 2.0 to strike a balance between open contributions and allowing you to use the software however you want.
Contributing
Please refer to contribution guidelines and internal concepts docs for more information.
Acknowledgement
Special thanks to all the contributors who have propelled GreptimeDB forward. For a complete list of contributors, please refer to AUTHOR.md.
- GreptimeDB uses Apache Arrow™ as the memory model and Apache Parquet™ as the persistent file format.
- GreptimeDB's query engine is powered by Apache Arrow DataFusion™.
- Apache OpenDAL™ gives GreptimeDB a very general and elegant data access abstraction layer.
- GreptimeDB's meta service is based on etcd.