* feat/manual-compaction-parallelism:
### Add Parallelism Support to Compaction Requests
- **`Cargo.lock` & `Cargo.toml`**: Updated `greptime-proto` dependency to a new revision.
- **`flush_compact_table.rs`**: Enhanced `parse_compact_params` to support a new `parallelism` parameter, allowing users to
specify the level of parallelism for table compaction.
- **`handle_compaction.rs`**: Integrated `parallelism` into the compaction scheduling process, defaulting to 1 if not
specified.
- **`request.rs` & `region_request.rs`**: Modified `CompactRequest` to include `parallelism`, with logic to handle unspecifie
values.
- **`requests.rs`**: Updated `CompactTableRequest` structure to include an optional `parallelism` field.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat/manual-compaction-parallelism:
### Commit Message
Enhance Compaction Request Handling
- **`flush_compact_table.rs`**:
- Renamed `parse_compact_params` to `parse_compact_request`.
- Introduced `DEFAULT_COMPACTION_PARALLELISM` constant.
- Updated parsing logic to handle keyword arguments for `strict_window` and `regular` compaction types, including `parallelism` and `window`.
- Modified tests to reflect changes in parsing logic and default parallelism handling.
- **`request.rs`**:
- Updated `parallelism` handling in `RegionRequestBody::Compact` to use the new default value.
- **`requests.rs`**:
- Changed `CompactTableRequest` to use a non-optional `parallelism` field with a default value of 1.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat/manual-compaction-parallelism:
### Update `flush_compact_table.rs` Parameter Validation
- Modified parameter validation in `flush_compact_table.rs` to restrict the maximum number of parameters from 4 to 3 in the `parse_compact_request` function.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat/manual-compaction-parallelism:
Update `greptime-proto` dependency
- Updated the `greptime-proto` dependency to a new revision in both `Cargo.lock` and `Cargo.toml`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat: add updated_on to tablemeta with a default of created_on
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* feat: support the update_on on alter procedure
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* feat: add updated_on into information_schema.tables
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* fix: make sqlness happy
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* test: add test case for tablemeta update
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* fix: fix failing test for ALTER TABLE
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* feat: use created_on as default for updated_on when missing
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
---------
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* feat: struct value
Signed-off-by: Ning Sun <sunning@greptime.com>
* feat: update for proto module
* feat: wip struct type
* feat: implement more vector operations
* feat: make datatype and api
* feat: reoslve some compilation issues
* feat: resolve all compilation issues
* chore: format update
* test: resolve tests
* test: test and refactor value-to-pb
* feat: add more tests and fix for value types
* chore: remove dbg
* feat: test and fix iterator
* fix: resolve struct_type issue
* refactor: use vec for struct items
* chore: update proto to main branch
* refactor: address some of review issues
* refactor: update for further review
* Add validation on new methods
* feat: update struct/list json serialization
* refactor: reimplement get in struct_vector
* refactor: struct vector functions
* refactor: fix lint issue
* refactor: address review comments
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
* feat: Add support for TWCS time window hints in insert operations
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat: set system events table time window to 1d
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
refactor: extract the common codes of creating proto ColumnSchema and Row to helper functions
fix: explicitly set the follower max sequence when finding extension ranges to avoid potential concurrency hazard
Signed-off-by: luofucong <luofc@foxmail.com>
* feat: add `SET DEFAULT` syntax
Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>
* test: add `CURRENT_TIMESTAMP()` as default value for `SET DEFAULT` syntax
Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>
* refactor: Make the error types more precise.
Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>
* chore: a minor error display enchancement for `SET DEFAULT`
Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>
* refactor: Using `MODIFY COLUMN` for `DROP/SET DEFUALT`
Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>
* chore: update `greptime-proto`
Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>
---------
Signed-off-by: Yihai Lin <yihai-lin@foxmail.com>
* feat: supports more db options
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: tests
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: use btree map for consistent results
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: adds compaction keys into valid db options
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* wip
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
### Commit Message
Enhance DDL Module Accessibility and Refactor `verify_alter` Function
- **`statement.rs`**: Made the `ddl` module public to enhance accessibility.
- **`ddl.rs`**:
- Made `NAME_PATTERN_REG` public for broader usage.
- Refactored `verify_alter` function to be a standalone public function, improving modularity and reusability.
- Made `parse_partitions` function public to allow external access.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
### Add Parquet Writer and Enhance Row Modifier
- **Add Parquet Writer Module**: Introduced a new module `parquet_writer.rs` to bridge `opendal` `Writer` with `parquet` `AsyncFileWriter`.
- **Enhance Row Modifier**: Updated `RowModifier` to use `Default` trait and made `fill_internal_columns` a public static method in `row_modifier.rs`.
- **Expose Internal Structures**: Made `RowsIter`, `RowIter`, `TablesBuilder`, and `TableBuilder` structs public in `row_modifier.rs` and `prom_row_builder.rs`.
- **Update Metric Engine**: Changed `RowModifier` instantiation to use `default()` in `engine.rs`.
- **Modify Table Options Handling**: Added `fill_table_options_for_create` function in `insert.rs` to handle table options based on `AutoCreateTableType`.
- **Make Constants Public**: Changed `DEFAULT_ROW_GROUP_SIZE` to public in `parquet.rs`.
- **Expose Functions**: Made `extract_add_columns_expr` public in `expr_helper.rs` and `AutoCreateTableType` public in `insert.rs`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
### Commit Message
Enhance HTTP Server and Prometheus Integration
- **`http.rs`**: Made `extractor` module public to allow external access.
- **`prom_store.rs`**: Refactored `decode_remote_write_request` to return `TablesBuilder` and adjusted logic for processing requests based on pipeline usage.
- **`lib.rs`**: Made `metrics` module public for broader accessibility.
- **`prom_row_builder.rs`**: Exposed `tables` field in `TablesBuilder` for external manipulation.
- **`proto.rs`**: Changed visibility of `table_data` in `PromWriteRequest` to `pub(crate)` for internal module access.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
### Add Accessor Methods for Managers and Executors
- **`src/frontend/src/instance.rs`**: Added accessor methods for `NodeManagerRef`, `PartitionRuleManagerRef`, `CacheInvalidatorRef`, and `ProcedureExecutorRef` to the `Instance` struct.
- **`src/operator/src/insert.rs`**: Introduced methods to access `NodeManagerRef` and `PartitionRuleManagerRef` in the `Inserter` struct.
- **`src/operator/src/statement.rs`**: Added methods to retrieve `ProcedureExecutorRef` and `CacheInvalidatorRef` in the `StatementExecutor` struct.
### Change HashMap Implementation
- **`src/servers/src/prom_row_builder.rs`**: Replaced `ahash::HashMap` with `std::collections::HashMap`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
Refactor table option handling in `insert.rs`
- Replaced `Vec` with `HashMap` for `table_options` to improve efficiency.
- Extracted logic for filling table options into a new function `fill_table_options_for_create`.
- Modified `fill_table_options_for_create` to return the engine name based on `create_type`.
- Simplified the insertion of table options into `create_table_expr` by using `extend` method.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
Refactor `insert.rs` to separate engine name logic from table options
- Updated `Inserter` implementation to determine `engine_name` separately from `fill_table_options_for_create`.
- Modified `fill_table_options_for_create` to no longer return an engine name, focusing solely on populating table options.
- Adjusted logic to set `engine_name` based on `AutoCreateTableType`, using `METRIC_ENGINE_NAME` for logical tables and `default_engine()` otherwise.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
