* feat: initial support for __schema__ in label values
* feat: filter database with matches
* refactor: skip unnecessary check
* fix: resolve schema matcher in label values
* test: add a test case for table not exists
* refactor: add matchop check on db label
* chore: merge main
* feat: supports null reponse format for http API
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: license header and assertion
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: in seconds
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: allow float number literal as step
Signed-off-by: Ning Sun <sunning@greptime.com>
* chore: switch to released version of promql parser
Signed-off-by: Ning Sun <sunning@greptime.com>
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
* fix: use `limit` params in jaeger http
Signed-off-by: zyy17 <zyylsxm@gmail.com>
* refactor: only parse `max_duration` and `min_duration` when it's not empty
Signed-off-by: zyy17 <zyylsxm@gmail.com>
* fix: handle the input for empty `limit` string
Signed-off-by: zyy17 <zyylsxm@gmail.com>
* fix: missing the fileter for `service_name`
Signed-off-by: zyy17 <zyylsxm@gmail.com>
* test: fix ci errors
Signed-off-by: zyy17 <zyylsxm@gmail.com>
* fix: incorrect behavior of find_traces
Signed-off-by: zyy17 <zyylsxm@gmail.com>
* fix: the logic of `find_traces()`
The correct logic should be:
1. Get all trace ids that match the filters;
2. Get all traces that match the trace ids from the previous query;
Signed-off-by: zyy17 <zyylsxm@gmail.com>
* fix: integration test errors
Signed-off-by: zyy17 <zyylsxm@gmail.com>
* refactor: add `empty_string_as_none`
Signed-off-by: zyy17 <zyylsxm@gmail.com>
* refactor: refine naming
Signed-off-by: zyy17 <zyylsxm@gmail.com>
---------
Signed-off-by: zyy17 <zyylsxm@gmail.com>
* chore/enable-flight-encoder:
### Add Flight Compression Support
- **Configuration Updates**:
- Added `grpc.flight_compression` option to `config/config.md`, `config/datanode.example.toml`, and `config/frontend.example.toml` to specify compression modes for Arrow IPC service.
- **Code Enhancements**:
- Updated `FlightEncoder` in `src/common/grpc/src/flight.rs` to support compression modes.
- Modified `RegionServer` and `DatanodeBuilder` in `src/datanode/src/datanode.rs` and `src/datanode/src/region_server.rs` to handle `FlightCompression`.
- Integrated `FlightCompression` in `src/servers/src/grpc.rs` and `src/servers/src/grpc/flight.rs` to manage compression settings.
- **Testing and Integration**:
- Updated test utilities and integration tests in `tests-integration/src/grpc/flight.rs` and `tests-integration/src/test_util.rs` to include `FlightCompression`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore/enable-flight-encoder:
### Enable Compression in FlightClient
- **`client.rs`**: Updated `make_flight_client` to accept `send_compression` and `accept_compression` parameters, enabling Zstd compression for sending and receiving messages.
- **`client_manager.rs`**: Modified `datanode` method to pass compression settings from `ChannelConfig` to `RegionRequester`.
- **`database.rs`**: Adjusted calls to `make_flight_client` to include compression parameters.
- **`region.rs`**: Updated `RegionRequester` to store and utilize compression settings.
- **`frontend.rs`**: Configured `ChannelConfig` to enable compression based on options.
- **`channel_manager.rs`**: Added `send_compression` and `accept_compression` fields to `ChannelConfig` with default values and updated tests accordingly.
Signed-off-by: Lei, HUANG <lhuang@greptime.com>
* chore/enable-flight-encoder:
### Update Compression Defaults and Documentation
- **Configuration Files**: Updated `datanode.example.toml` and `frontend.example.toml` to include a default setting comment for `flight_compression`, specifying it defaults to `none`.
- **gRPC Server Code**: Modified `grpc.rs` to set `None` as the default for `FlightCompression` instead of `ArrowIpc`.
Signed-off-by: Lei, HUANG <lhuang@greptime.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Signed-off-by: Lei, HUANG <lhuang@greptime.com>
* fix unit tests
* fix: sqlness
* fix/default-time-window:
## Add Helper Functions and Enhance Compaction Tests
- **Refactor Compaction Logic**: Introduced helper functions `flush` and `compact` in `compaction_test.rs` to streamline compaction operations.
- **Enhance Compaction Tests**: Added a new test `test_infer_compaction_time_window` in `compaction_test.rs` to verify compaction time window inference.
- **Testing Improvements**: Added `#[cfg(test)]` attribute to `new_multi_partitions` in `time_partition.rs` to ensure it's only included in test builds.
* fix/default-time-window:
- **Refactor `TimePartition` Struct**: Removed unnecessary comments regarding `time_range` in `time_partition.rs`.
- **Enhance `TimePartitions` Functionality**: Added a method `part_duration_or_default` to provide a default partition duration in `time_partition.rs`.
- **Update SQL Test Cases**: Modified SQL operations and expected results in `scan_big_varchar.result` and `scan_big_varchar.sql` to reflect changes in data manipulation logic.
* fix/default-time-window:
### Update Time Partition Default Duration
- **Refactor Default Duration**: Introduced `INITIAL_TIME_WINDOW` constant to define the default time window duration as `Duration::from_days(1)`. This change replaces multiple instances of the hardcoded default duration across the `time_partition.rs` file.
- **Files Affected**: `time_partition.rs`
* fix/default-time-window:
## Update Partition Duration Handling
- **`time_partition.rs`**: Refactored `part_duration` to be non-optional, removing `Option` wrapper. Updated logic to use `unwrap_or` with `INITIAL_TIME_WINDOW` where necessary. Adjusted related methods and tests to accommodate this change.
- **`version.rs` (memtable and region)**: Updated handling of `part_duration` to align with changes in `time_partition.rs`, ensuring consistent use of non-optional `Duration`.
* fix/default-time-window:
### Improve Error Context in `time_partition.rs`
- Enhanced error context message in `time_partition.rs` to provide clearer information on partition time range issues, including bucket size details.
Signed-off-by: Lei, HUANG <lhuang@greptime.com>
---------
Signed-off-by: Lei, HUANG <lhuang@greptime.com>
* feat: pipeline recognize hints from exec
* chore: rename and add test
* chore: minor improve
* chore: rename and add comments
* fix: typos
* feat: add initial impl for vrl processor
* chore: update processors to allow vrl process
* feat: pipeline recognize hints from exec
* chore: rename and add test
* chore: minor improve
* chore: rename and add comments
* fix: typos
* chore: remove unnecessory clone fn
* chore: group metrics
* chore: use struct in transform output enum
* test: add test for vrl
* fix: leaked conflicts
* chore: merge branch code & add check in compile
* fix: check condition
* fix: check auto-transform timeindex
* chore: support table_suffix in hint
* chore: add test for table suffix in vrl hint
* refactor: change context_opt to a struct
* chore: support shared pipeline under catalog with compatibility
* test: add test for cross schema ref
* chore: use empty string schema by default
* chore: remove unwrap in the patch
* fix: df check
* feat/lossy-string-validation-in-prom-remote-write:
### Commit Message
#### Refactor Prometheus Validation Mode
- **Replace `is_strict_mode` with `PromValidationMode` Enum:**
- Updated `HttpOptions` and related structures to use `PromValidationMode` enum instead of the boolean `is_strict_mode`.
- Modified functions and tests to accommodate the new enum, ensuring flexible validation modes (`Strict`, `Lossy`, `Unchecked`).
- Affected files: `server.rs`, `prom_decode.rs`, `http.rs`, `prom_store.rs`, `prom_row_builder.rs`, `proto.rs`, `prom_store_test.rs`, `test_util.rs`, `http.rs`.
- **Enhance UTF-8 String Decoding:**
- Introduced `decode_string` function to handle UTF-8 string decoding based on the selected `PromValidationMode`.
- Affected files: `proto.rs`, `prom_row_builder.rs`.
This refactor improves the flexibility and clarity of Prometheus request handling by allowing different validation strategies.
* feat/lossy-string-validation-in-prom-remote-write:
- **Add Prometheus Validation Mode Configuration:**
- Updated `config/config.md`, `config/frontend.example.toml`, and `config/standalone.example.toml` to include `http.prom_validation_mode` setting for Prometheus remote write requests.
- **Enhance Benchmarking for Prometheus Requests:**
- Modified `src/servers/benches/prom_decode.rs` to benchmark different Prometheus validation modes (`Strict`, `Lossy`, `Unchecked`).
- **Implement and Test String Decoding:**
- Added `decode_string` function and comprehensive tests in `src/servers/src/proto.rs` to handle string decoding with different validation modes.
* feat/lossy-string-validation-in-prom-remote-write:
### Add Histogram Buckets to Metrics
- **Files Modified**: `src/servers/src/metrics.rs`
- **Key Changes**:
- Added specific histogram buckets to `METRIC_MYSQL_QUERY_TIMER`, `METRIC_POSTGRES_QUERY_TIMER`, and `METRIC_SERVER_GRPC_PROM_REQUEST_TIMER` to enhance granularity in query elapsed time metrics.
* feat/lossy-string-validation-in-prom-remote-write:
### Update Prometheus Validation Mode Default
- **Config Documentation**: Updated the default description for `http.prom_validation_mode` to indicate that "strict" is the default option in `config.md`, `frontend.example.toml`, and `standalone.example.toml`.
- **HTTP Server Implementation**: Changed the default `prom_validation_mode` to `PromValidationMode::Strict` in `src/servers/src/http.rs`.
* feat/lossy-string-validation-in-prom-remote-write:
**Commit Message:**
Update Prometheus Validation Mode to Strict
- Changed `http.prom_validation_mode` from `unchecked` to `strict` in `config.md`, `frontend.example.toml`, and
`standalone.example.toml` to enforce strict validation of Prometheus remote write requests.
