mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-10-29 15:06:59 +00:00
feat(mito): bulk insert request handling on datanode (#5831)
* wip: implement basic request handling * feat/bulk-insert: ### Add Error Handling and Enhance Bulk Insert Functionality - **Error Handling**: Introduced a new error variant `ConvertDataType` in `error.rs` to handle conversion failures from `ConcreteDataType` to `ColumnDataType`. - **Bulk Insert Enhancements**: - Updated `WorkerRequest::BulkInserts` in `request.rs` to include metadata and sender. - Implemented `handle_bulk_inserts` in `worker.rs` to process bulk insert requests with region metadata. - Added functions `region_metadata_to_column_schema` and `record_batch_to_rows` in `handle_bulk_insert.rs` for schema conversion and row processing. - **API Changes**: Modified `RegionBulkInsertsRequest` in `region_request.rs` to include `region_id`. Files affected: `error.rs`, `request.rs`, `worker.rs`, `handle_bulk_insert.rs`, `region_request.rs`. * feat/bulk-insert: **Enhance Error Handling and Add Unit Tests** - Improved error handling in `record_batch_to_rows` function within `handle_bulk_insert.rs` by returning `Result` and handling errors with `context`. - Added unit tests for `region_metadata_to_column_schema` and `record_batch_to_rows` functions in `handle_bulk_insert.rs` to ensure correct functionality and error handling. * chore: update proto version * feat/bulk-insert: - **Refactor Error Handling**: Updated error handling in `error.rs` by modifying the `ConvertDataType` error handling. - **Improve Logging and Error Reporting**: Enhanced logging and error reporting in `worker.rs` by adding error messages for missing region metadata. - **Add New Error Type**: Introduced `DecodeArrowIpc` error in `metadata.rs` to handle Arrow IPC decoding failures. - **Handle Arrow IPC Decoding**: Updated `region_request.rs` to handle Arrow IPC decoding errors using the new `DecodeArrowIpc` error type. * chore: update proto version * feat/bulk-insert: Refactor `handle_bulk_insert.rs` to simplify row construction - Removed the mutable `current_row` vector and refactored `row_at` function to return a new vector directly. - Updated `record_batch_to_rows` to utilize the refactored `row_at` function for constructing rows. * feat/bulk-insert: ### Commit Summary **Enhancements in Region Server Request Handling** - Updated `region_server.rs` to include `RegionRequest::BulkInserts(_)` in the `RegionChange::Ingest` category, improving the handling of bulk insert operations. - Refined the categorization of region requests to ensure accurate mapping to `RegionChange` actions.
This commit is contained in:
@@ -902,7 +902,9 @@ impl RegionServerInner {
|
||||
RegionChange::Register(attribute)
|
||||
}
|
||||
RegionRequest::Close(_) | RegionRequest::Drop(_) => RegionChange::Deregisters,
|
||||
RegionRequest::Put(_) | RegionRequest::Delete(_) => RegionChange::Ingest,
|
||||
RegionRequest::Put(_) | RegionRequest::Delete(_) | RegionRequest::BulkInserts(_) => {
|
||||
RegionChange::Ingest
|
||||
}
|
||||
RegionRequest::Alter(_)
|
||||
| RegionRequest::Flush(_)
|
||||
| RegionRequest::Compact(_)
|
||||
|
||||
Reference in New Issue
Block a user