diff --git a/config/config.md b/config/config.md index 6faae09c347..6898973b214 100644 --- a/config/config.md +++ b/config/config.md @@ -140,6 +140,7 @@ | `procedure.retry_delay` | String | `500ms` | Initial retry delay of procedures, increases exponentially | | `procedure.max_running_procedures` | Integer | `128` | Max running procedures.
The maximum number of procedures that can be running at the same time.
If the number of running procedures exceeds this limit, the procedure will be rejected. | | `query` | -- | -- | The query engine options. | +| `query.experimental_native_promql` | Bool | `false` | Whether to enable experimental native PromQL execution.
Default: false | | `query.parallelism` | Integer | `0` | Parallelism of the query engine.
Default to 0, which means the number of CPU cores. | | `query.memory_pool_size` | String | 0 | Memory pool size for query execution operators (aggregation, sorting, join).
Supports absolute size (e.g., "2GB", "4GB") or percentage of system memory (e.g., "20%").
Setting it to 0 disables the limit. Defaults to 0 (unlimited); this example sets it to `50%`.
When this limit is reached, queries will fail with ResourceExhausted error.
NOTE: This does NOT limit memory used by table scans. | | `query.experimental_memory_pool_policy` | String | `greedy` | Experimental memory pool allocation policy:
- "greedy" (default): first-come-first-served allocation; preserves current behavior.
- "fair": divides available memory among spillable operators and may spill earlier.
Only effective when `memory_pool_size` is bounded (>0). | @@ -383,6 +384,7 @@ | `meta_client.metadata_cache_ttl` | String | `10m` | TTL of the metadata cache. | | `meta_client.metadata_cache_tti` | String | `5m` | -- | | `query` | -- | -- | The query engine options. | +| `query.experimental_native_promql` | Bool | `false` | Whether to enable experimental native PromQL execution.
Default: false | | `query.parallelism` | Integer | `0` | Parallelism of the query engine.
Default to 0, which means the number of CPU cores. | | `query.allow_query_fallback` | Bool | `false` | Whether to allow query fallback when push down optimize fails.
Default to false, meaning when push down optimize failed, return error msg | | `query.memory_pool_size` | String | 0 | Memory pool size for query execution operators (aggregation, sorting, join).
Supports absolute size (e.g., "4GB", "8GB") or percentage of system memory (e.g., "30%").
Setting it to 0 disables the limit. Defaults to 0 (unlimited); this example sets it to `50%`.
When this limit is reached, queries will fail with ResourceExhausted error.
NOTE: This does NOT limit memory used by table scans (only applies to datanodes). | @@ -613,6 +615,7 @@ | `wal.max_unpersisted_age` | String | `8s` | The age of the oldest unpersisted entry at which new appends stall until an upload completes, in `enqueued` mode.
**It's only used when the provider is `experimental_object_store` and `ack_mode` is `enqueued`**. | | `wal.on_corrupted_segment` | String | `skip` | What a read does with a segment that still does not decode after a second fetch, because its checksum does not match or its content disagrees with its footer entry.
- `skip`: the segment is skipped and recorded as a WAL hole of its region, a metric is incremented and a warning is logged; the other regions of the object are unaffected (the default).
- `fail`: the read fails, so the region does not open.
**It's only used when the provider is `experimental_object_store`**. | | `query` | -- | -- | The query engine options. | +| `query.experimental_native_promql` | Bool | `false` | Whether to enable experimental native PromQL execution.
Default: false | | `query.parallelism` | Integer | `0` | Parallelism of the query engine.
Default to 0, which means the number of CPU cores. | | `query.memory_pool_size` | String | 0 | Memory pool size for query execution operators (aggregation, sorting, join).
Supports absolute size (e.g., "2GB", "4GB") or percentage of system memory (e.g., "20%").
Setting it to 0 disables the limit. Defaults to 0 (unlimited); this example sets it to `50%`.
When this limit is reached, queries will fail with ResourceExhausted error.
NOTE: This does NOT limit memory used by table scans. | | `query.experimental_memory_pool_policy` | String | `greedy` | Experimental memory pool allocation policy:
- "greedy" (default): first-come-first-served allocation; preserves current behavior.
- "fair": divides available memory among spillable operators and may spill earlier.
Only effective when `memory_pool_size` is bounded (>0). | @@ -799,6 +802,7 @@ | `tracing` | -- | -- | The tracing options. Only effect when compiled with `tokio-console` feature. | | `tracing.tokio_console_addr` | String | Unset | The tokio console address. | | `query` | -- | -- | -- | +| `query.experimental_native_promql` | Bool | `false` | Whether to enable experimental native PromQL execution.
Default: false | | `query.parallelism` | Integer | `1` | Parallelism of the query engine for query sent by flownode.
Default to 1, so it won't use too much cpu or memory | | `query.memory_pool_size` | String | 0 | Memory pool size for query execution operators (aggregation, sorting, join).
Supports absolute size (e.g., "1GB", "2GB") or percentage of system memory (e.g., "20%").
Setting it to 0 disables the limit. Defaults to 0 (unlimited); this example sets it to `50%`.
When this limit is reached, queries will fail with ResourceExhausted error.
NOTE: This does NOT limit memory used by table scans. | | `query.experimental_memory_pool_policy` | String | `greedy` | Experimental memory pool allocation policy:
- "greedy" (default): first-come-first-served allocation; preserves current behavior.
- "fair": divides available memory among spillable operators and may spill earlier.
Only effective when `memory_pool_size` is bounded (>0). | diff --git a/config/datanode.example.toml b/config/datanode.example.toml index 2d72f795189..40eaff54f3b 100644 --- a/config/datanode.example.toml +++ b/config/datanode.example.toml @@ -326,6 +326,10 @@ overwrite_entry_start_id = false ## The query engine options. [query] + +## Whether to enable experimental native PromQL execution. +## Default: false +#+ experimental_native_promql = false ## Parallelism of the query engine. ## Default to 0, which means the number of CPU cores. parallelism = 0 diff --git a/config/flownode.example.toml b/config/flownode.example.toml index fab31fed510..4827e576258 100644 --- a/config/flownode.example.toml +++ b/config/flownode.example.toml @@ -151,6 +151,10 @@ default_ratio = 1.0 #+ tokio_console_addr = "127.0.0.1" [query] + +## Whether to enable experimental native PromQL execution. +## Default: false +#+ experimental_native_promql = false ## Parallelism of the query engine for query sent by flownode. ## Default to 1, so it won't use too much cpu or memory parallelism = 1 diff --git a/config/frontend.example.toml b/config/frontend.example.toml index c42d254671c..8f0122a04ff 100644 --- a/config/frontend.example.toml +++ b/config/frontend.example.toml @@ -359,6 +359,10 @@ metadata_cache_tti = "5m" ## The query engine options. [query] + +## Whether to enable experimental native PromQL execution. +## Default: false +#+ experimental_native_promql = false ## Parallelism of the query engine. ## Default to 0, which means the number of CPU cores. parallelism = 0 diff --git a/config/standalone.example.toml b/config/standalone.example.toml index 82fb3d2c489..9964653564d 100644 --- a/config/standalone.example.toml +++ b/config/standalone.example.toml @@ -537,6 +537,10 @@ max_running_procedures = 128 ## The query engine options. [query] + +## Whether to enable experimental native PromQL execution. +## Default: false +#+ experimental_native_promql = false ## Parallelism of the query engine. ## Default to 0, which means the number of CPU cores. parallelism = 0 diff --git a/src/query/src/options.rs b/src/query/src/options.rs index a49021f2f15..239c485070a 100644 --- a/src/query/src/options.rs +++ b/src/query/src/options.rs @@ -110,6 +110,9 @@ pub struct QueryOptions { /// - `fair`: Fair divides memory available after unspillable reservations /// evenly among spillable reservations and may trigger earlier spills. pub experimental_memory_pool_policy: QueryMemoryPoolPolicy, + /// Enables the experimental native PromQL execution path for compatible + /// instant queries over shared Metric raw series. Default: `false`. + pub experimental_native_promql: bool, } #[allow(clippy::derivable_impls)] @@ -125,6 +128,7 @@ impl Default for QueryOptions { experimental_spill_max_temp_directory_size: ReadableSize::gb(1), experimental_spill_compression: QuerySpillCompression::Uncompressed, experimental_memory_pool_policy: QueryMemoryPoolPolicy::Greedy, + experimental_native_promql: false, } } } diff --git a/tests-integration/tests/http.rs b/tests-integration/tests/http.rs index f3b439e19bc..bb9216e6734 100644 --- a/tests-integration/tests/http.rs +++ b/tests-integration/tests/http.rs @@ -3512,6 +3512,7 @@ experimental_spill_mode = "default" experimental_spill_max_temp_directory_size = "1GiB" experimental_spill_compression = "uncompressed" experimental_memory_pool_policy = "greedy" +experimental_native_promql = false [memory] enable_heap_profiling = true