mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-01-11 07:42:54 +00:00
6da633e70d
* feat/kill-process: ### Add Cancellation Support and Enhance Process Management - **Cancellation Handle Implementation**: Introduced `CancellationHandle` in `cancellation_handle.rs` to facilitate cancellation of futures and streams. - **Process Management Enhancements**: - Updated `ProcessManager` in `process_manager.rs` to support cancellable processes using `CancellableProcess`. - Added `kill_process` method for terminating processes. - **Stream Wrapper Update**: - Replaced `StreamWrapper` with `CancellableStreamWrapper` in `stream_wrapper.rs` and `instance.rs` to handle stream cancellation. - **Error Handling**: - Added `StreamCancelled` error variant in `error.rs` to handle stream cancellation scenarios. - **gRPC Handler Update**: - Added `kill_process` gRPC method in `frontend_grpc_handler.rs` to allow external process termination. - **Dependency Updates**: - Updated `Cargo.lock` and `Cargo.toml` to include `common-base` and `tokio-util`. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: **Enhancements and Bug Fixes** - **Dependency Update**: Updated `greptime-proto` dependency in `Cargo.lock` and `Cargo.toml` to a new revision. - **Error Handling Improvements**: - Modified error variants in `src/catalog/src/error.rs` and `src/common/frontend/src/error.rs` to improve error messages and handling. - Added `FrontendNotFound` error variant for better error specificity. - **Process Management Enhancements**: - Updated `ProcessManager` in `src/catalog/src/process_manager.rs` to include `kill_process` functionality with server address validation. - Enhanced `FrontendClient` trait in `src/common/frontend/src/selector.rs` to support `kill_process` requests. - **gRPC Handler Update**: - Refactored `FrontendGrpcHandler` in `src/servers/src/grpc/frontend_grpc_handler.rs` to handle `kill_process` requests asynchronously and return process status. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: ### Add Kill Process Functionality - **`Cargo.lock`, `Cargo.toml`**: Added `common-frontend` as a dependency. - **`server.rs`, `builder.rs`, `instance.rs`**: Updated `FrontendInvoker` and `FrontendBuilder` to support process management. - **`error.rs`**: Introduced `InvalidProcessId` error for handling invalid process IDs. - **`statement.rs`, `kill.rs`**: Implemented `execute_kill` method in `StatementExecutor` to handle the `KILL` statement. - **`parser.rs`, `statement.rs`**: Updated SQL parser to recognize and parse the `KILL` statement. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: ## Add Cancellation Support to Query Execution - **`process_manager.rs`**: Updated `CancellationHandle` initialization to use `default()` method. - **`cancellation_handle.rs`**: Implemented `Debug` trait for `CancellationHandle` and added `Cancellation` and `CancellableFuture` structs to support cancellable futures. - **`error.rs`**: Introduced `Cancelled` error variant to handle query cancellations. - **`instance.rs`**: Integrated `CancellableFuture` to manage query execution with cancellation support. - **`stream_wrapper.rs`**: Modified `CancellableStreamWrapper` to use the new `waker()` method for cancellation handling. - **`statement.rs`**: Added `#[allow(clippy::too_many_arguments)]` to `StatementExecutor::new` to suppress clippy warnings. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: - **Add `MetaClientMissing` Error**: Introduced a new error variant `MetaClientMissing` in `error.rs` to handle missing meta client scenarios. - **Refactor Cancellation Handling**: Merged `cancellation_handle.rs` into `cancellation.rs` and updated related logic in `process_manager.rs`, `instance.rs`, and `stream_wrapper.rs`. - **Enhance Process Management**: Improved process management logic in `process_manager.rs` to handle process cancellation more effectively. - **Update Tests**: Added and updated tests in `cancellation.rs` and `stream_wrapper.rs` to cover new cancellation logic and error handling. - **Cargo.toml Update**: Adjusted workspace settings in `Cargo.toml` for `common-frontend`. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: - **Add Tests for Process Management**: Introduced multiple async tests in `process_manager.rs` to verify query registration, deregistration, cancellation, and process killing functionalities. - **Update Error Message in SQL Parser**: Modified the expected error message in `parser.rs` to clarify the expected token as a "process id string literal". Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: ### Add Process Count Metrics to Catalog - **`metrics.rs`**: Introduced a new metric `PROCESS_LIST_COUNT` to track the count of running processes per catalog using `IntGaugeVec`. - **`process_manager.rs`**: Updated `CancellableProcess` to increment and decrement `PROCESS_LIST_COUNT` upon creation and destruction, respectively. Added a `Drop` implementation for `CancellableProcess` to handle metric updates. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: ### Fix process removal logic in `process_manager.rs` - Corrected the condition for removing an entry from the catalog in `ProcessManager` by using `o.get()` instead of `o.get_mut()`. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: - **Error Handling Improvements**: - Updated status codes for `Error::FrontendNotFound` and `Error::MetaClientMissing` to `StatusCode::Unexpected` in `src/catalog/src/error.rs`. - Changed `InvokeFrontend` error display message and status code in `src/common/frontend/src/error.rs`. - Added `ProcessManagerMissing` error in `src/operator/src/error.rs` and updated its handling in `src/operator/src/statement/kill.rs`. - **Process Management Enhancements**: - Added documentation for `ProcessManager` and `register_query` in `src/catalog/src/process_manager.rs`. - Modified `kill_process` response handling in `src/servers/src/grpc/frontend_grpc_handler.rs`. - **Cancellation Logic Update**: - Improved cancellation logic in `src/common/base/src/cancellation.rs` to use `compare_exchange` for atomic operations. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: ### Add Process Kill Count Metric and Refactor Cancellation Handle - **Metrics Update**: Added a new metric `PROCESS_KILL_COUNT` in `metrics.rs` to track the count of completed kill process requests per catalog. - **Refactor Cancellation Handle**: Renamed `cancellation_handler` to `cancellation_handle` across multiple files for consistency: - `process_manager.rs` - `instance.rs` - `stream_wrapper.rs` - **Process Management**: Updated process management logic in `process_manager.rs` to increment the `PROCESS_KILL_COUNT` metric upon successful process termination. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: Update metric description in `metrics.rs` - Changed the description of `PROCESS_KILL_COUNT` to reflect the count of killed processes instead of running processes in `metrics.rs`. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/kill-process: Update `greptime-proto` Dependency and Fix Response Field - **Updated Dependency**: Changed the `greptime-proto` Git revision in `Cargo.lock` and `Cargo.toml` to `f0913f1`. - **Code Fix**: Modified `frontend_grpc_handler.rs` to correct the response field from `found` to `success` in `KillProcessResponse`. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> --------- Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Real-Time & Cloud-Native Observability Database
for metrics, logs, and traces
Delivers sub-second querying at PB scale and exceptional cost efficiency from edge to cloud.
- Introduction
- ⭐ Key Features
- Quick Comparison
- Architecture
- Try GreptimeDB
- Getting Started
- Build From Source
- Tools & Extensions
- Project Status
- Community
- License
- Commercial Support
- Contributing
- Acknowledgement
Introduction
GreptimeDB is an open-source, cloud-native database purpose-built for the unified collection and analysis of observability data (metrics, logs, and traces). Whether you’re operating on the edge, in the cloud, or across hybrid environments, GreptimeDB empowers real-time insights at massive scale — all in one system.
Features
| Feature | Description |
|---|---|
| Unified Observability Data | Store metrics, logs, and traces as timestamped, contextual wide events. Query via SQL, PromQL, and streaming. |
| High Performance & Cost Effective | Written in Rust, with a distributed query engine, rich indexing, and optimized columnar storage, delivering sub-second responses at PB scale. |
| Cloud-Native Architecture | Designed for Kubernetes, with compute/storage separation, native object storage (AWS S3, Azure Blob, etc.) and seamless cross-cloud access. |
| Developer-Friendly | Access via SQL/PromQL interfaces, REST API, MySQL/PostgreSQL protocols, and popular ingestion protocols. |
| Flexible Deployment | Deploy anywhere: edge (including ARM/Android) or cloud, with unified APIs and efficient data sync. |
Learn more in Why GreptimeDB and Observability 2.0 and the Database for It.
Quick Comparison
| Feature | GreptimeDB | Traditional TSDB | Log Stores |
|---|---|---|---|
| Data Types | Metrics, Logs, Traces | Metrics only | Logs only |
| Query Language | SQL, PromQL, Streaming | Custom/PromQL | Custom/DSL |
| Deployment | Edge + Cloud | Cloud/On-prem | Mostly central |
| Indexing & Performance | PB-Scale, Sub-second | Varies | Varies |
| Integration | REST, SQL, Common protocols | Varies | Varies |
Performance:
Read more benchmark reports.
Architecture
- Read the architecture document.
- DeepWiki provides an in-depth look at GreptimeDB:
Try GreptimeDB
1. Live Demo
Experience GreptimeDB directly in your browser.
2. GreptimeCloud
Start instantly with a free cluster.
3. Docker (Local Quickstart)
docker pull greptime/greptimedb
docker run -p 127.0.0.1:4000-4003:4000-4003 \
-v "$(pwd)/greptimedb_data:/greptimedb_data" \
--name greptime --rm \
greptime/greptimedb:latest standalone start \
--http-addr 0.0.0.0:4000 \
--rpc-bind-addr 0.0.0.0:4001 \
--mysql-addr 0.0.0.0:4002 \
--postgres-addr 0.0.0.0:4003
Dashboard: http://localhost:4000/dashboard Full Install Guide
Troubleshooting:
- Cannot connect to the database? Ensure that ports
4000,4001,4002, and4003are not blocked by a firewall or used by other services. - Failed to start? Check the container logs with
docker logs greptimefor further details.
Getting Started
Build From Source
Prerequisites:
- Rust toolchain (nightly)
- Protobuf compiler (>= 3.15)
- C/C++ building essentials, including
gcc/g++/autoconfand glibc library (eg.libc6-devon Ubuntu andglibc-develon Fedora) - Python toolchain (optional): Required only if using some test scripts.
Build and Run:
make
cargo run -- standalone start
Tools & Extensions
- Kubernetes: GreptimeDB Operator
- Helm Charts: Greptime Helm Charts
- Dashboard: Web UI
- SDKs/Ingester: Go, Java, C++, Erlang, Rust, JS
- Grafana: Official Dashboard
Project Status
Status: Beta. GA (v1.0): Targeted for mid 2025.
- Being used in production by early adopters
- Stable, actively maintained, with regular releases (version info)
- Suitable for evaluation and pilot deployments
For production use, we recommend using the latest stable release.
If you find this project useful, a ⭐ would mean a lot to us!
Community
We invite you to engage and contribute!
License
GreptimeDB is licensed under the Apache License 2.0.
Commercial Support
Running GreptimeDB in your organization? We offer enterprise add-ons, services, training, and consulting. Contact us for details.
Contributing
- Read our Contribution Guidelines.
- Explore Internal Concepts and DeepWiki.
- Pick up a good first issue and join the #contributors Slack channel.
Acknowledgement
Special thanks to all contributors! See AUTHORS.md.
- Uses Apache Arrow™ (memory model)
- Apache Parquet™ (file storage)
- Apache Arrow DataFusion™ (query engine)
- Apache OpenDAL™ (data access abstraction)