Files
greptimedb/config/config.md
T
+9 a788cd977a chore: backport selected changes for 1.3.0-beta.2 (#9529)
* ci(query-regression): bump RUNNER_IMAGE_EPOCH to 7 (#9395)

ci(query-regression): bump RUNNER_IMAGE_EPOCH to 7 for image m-0xihbfzm5xpbxolybo6i

Signed-off-by: greptimedb-ci <greptimedb-ci@greptime.com>
Co-authored-by: greptimedb-ci <greptimedb-ci@greptime.com>
(cherry picked from commit 886e02f0dc)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): apply offset to subquery evaluation window (#9364)

* fix(promql): apply offset to subquery evaluation window

`prom_subquery_expr_to_plan` destructured `SubqueryExpr` without reading
`offset`, so `<subquery>[range:step] offset <d>` planned exactly the same
window as the un-offset form and silently returned data for the wrong time
range. The plain vector/matrix-selector paths already threaded the offset
through `selector_to_series_normalize_plan` and `RangeManipulate`.

Shift the inner evaluation window back by the offset and pass the offset to
the subquery's `RangeManipulate`, which maps the inner samples forward onto
the evaluation timeline before bucketing them into ranges. This matches
Prometheus, whose `evaluator.subqueryTimeRange` evaluates the inner
expression over `(start - offset - range, end - offset]` and whose
`evalSubquery` then hands the samples to the outer range-vector function as
a `MatrixSelector` that still carries the subquery offset. An offset on the
inner selector composes additively, as `subqueryTimes` documents.

`RangeManipulate`'s protobuf message has no offset field and recovers it on
decode from an immediately underlying `SeriesNormalize`. Since
`RangeManipulate` is commutative in `dist_plan` and can be pushed below a
`MergeScan`, insert that carrier node so the offset survives distributed
planning instead of decoding as zero.

Known divergence, unchanged by this commit: Prometheus anchors subquery step
points on absolute epoch multiples of the step, while GreptimeDB anchors them
on the evaluation start. The two agree whenever the offset is a multiple of
the subquery step; the added sqlness cases stay within that range.

Closes #9330

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Aarav <aaravsjadav@gmail.com>

* docs(promql): correct the subquery-offset rationale and pin the histogram shift

Follow-up to e132c267, which stated two things inaccurately and left one
path untested. No behaviour change beyond a conditional clone.

The `SeriesNormalize` carrier was described as existing "so the offset
survives distributed planning", implying standalone is unaffected. That is
wrong. `local_offset` is read on the substrait decode path
(`range_manipulate.rs`, `instant_manipulate.rs`), and GreptimeDB routes
PromQL plans through `MergeScan`/substrait in standalone too -- the
standalone results `promql/encode_substrait.result` and `precisions.result`
both show `MergeScan [is_placeholder=false, remote_input=[...]]`. Deleting
the carrier therefore empties subquery-with-offset results in standalone as
well, while `count_over_time_subquery_with_offset` keeps passing because it
asserts the pre-serialization plan. Say so at the node, so the next reader
does not remove it believing standalone is safe.

The carrier is also not semantically inert: `SeriesNormalizeStream::normalize`
shifts native histogram `start_timestamp` payloads by `offset`. On the
subquery path that runs on a computed inner result, on top of any offset the
inner selector already applied, and nothing pinned it. It is consistent --
`RangeManipulate` moves the millisecond time index forward by the same
`offset_ms` -- so the distance from a histogram's start timestamp to the
sample carrying it, which reset/rate detection reads, is invariant. Add
`subquery_offset_shifts_histogram_start_and_time_index_together`, which
executes the subquery's node stack over native histogram samples for a
zero and a non-zero inner offset. It cannot be a sqlness case: native
histograms are a struct column with no SQL type or literal
(`sql_data_type_to_concrete_data_type` rejects structs) and the sqlness
runner speaks only MySQL/Postgres, so such rows only arrive over
gRPC/remote-write v2.

The divergence note blamed sub-step offsets. Measured, that condition is too
narrow: an unaligned evaluation timestamp alone diverges, with no offset at
all -- `sum_over_time(fine[20s:10s])` at t=57 samples 47s and 57s here
against Prometheus's 40s and 50s. The real condition is that
`(start - offset)` is not a multiple of the subquery step, and it is
pre-existing: the committed `tql eval (359, 359, '1s')
sum_over_time(metric_total[60s:10s])` case is already an instance of it
(`359 - 60 + 10 = 309`). Reword the comment and the sqlness header
accordingly. A sub-step offset stays accepted: `foo[20s:10s] offset 5s` is
valid PromQL, erroring on it would be a regression and would not close the
gap anyway. This repo has no PromQL-compatibility documentation page, so
there is nowhere else to record it.

Also stop cloning the series key columns when there is no offset, and stop
the `offset 0s` sqlness comment implying its parser error is evidence about
this fix -- the same message comes from a `u32` overflow in
`promql-parser`'s shared duration check (`offset 9999999999d`).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Aarav <aaravsjadav@gmail.com>

* refactor(promql): trim subquery-offset comments and tests per review

Shorten the planner and sqlness comments to the semantic contract, build
`SeriesDivide` with a clone instead of an `Option`, and drop the
normalize.rs histogram test and the `offset 0s` parser case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bvab9gpXYjKyLN8Mo6JWWL
Signed-off-by: Aarav <aaravsjadav@gmail.com>

* fix(promql): record the subquery offset as the range fold offset

The subquery's `RangeManipulate` now shifts the payload timestamps by the
offset, so `predict_linear` over an offset subquery must recover the
evaluation time with that offset instead of 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Bvab9gpXYjKyLN8Mo6JWWL
Signed-off-by: Aarav <aaravsjadav@gmail.com>

---------

Signed-off-by: Aarav <aaravsjadav@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit d6474b961c)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: retry failed nightly release on following weekdays (#9394)

Previously the nightly release was scheduled only on Mondays, so a
failed run left users without nightly builds for a whole week.

Now the schedule triggers every weekday at 00:00 UTC, but the release
only proceeds when the latest published nightly release is older than
NIGHTLY_RELEASE_MAX_AGE_DAYS (5) days. The check runs in
allocate-runners before any EC2 runner is allocated, and gates all
schedule-driven jobs (including the Slack notification, so a skipped
nightly is silent). Tag pushes and manual dispatches are unaffected.

Signed-off-by: Ning Sun <sunning@greptime.com>
(cherry picked from commit a7f99b3425)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(client): yield Flight batches and affected rows without waiting for next message (#8918)

* fix(client): avoid Flight metrics lookahead stalls

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor(client): extract trailing Flight metrics task

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(client): synchronize trailing metrics and bound cancellation

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit abf1396c28)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): resolve dotted column names as unqualified columns (#9391)

* fix(promql): resolve dotted column names as unqualified columns

col(), From<&str>/From<String> for Column and string join keys go through
Column::from_qualified_name, which splits `service.name` into relation
`service` and column `name` and lowercases unquoted identifiers. Build
PromQL column references with Column::from_name / ident() instead.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(servers): resolve remote read matcher labels as unqualified columns

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): cover same-name columns differing in case and without()

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 218000e21b)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): align timestamp(), label_join and label_replace with Prometheus semantics (#9385)

* fix(promql): align timestamp() and label_join with Prometheus semantics

- timestamp() over a selector reports the selected sample's timestamp, without adding the offset.
- timestamp() over any other expression reports the evaluation time instead of the input value.
- label_join that overwrites an existing label rejects duplicate label sets at runtime.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): align label_replace and label_join edge cases with Prometheus

- label_replace leaves non-matching series unchanged instead of copying the source value into the destination label.
- label_replace may overwrite an existing label; duplicate label sets are rejected at runtime like label_join.
- label_join with no source labels removes the destination label, and an empty source label name is rejected.
- Empty label values produced by these functions are NULL, the same as an absent label.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): join absent labels as empty strings in label_join

concat_ws skips NULL arguments together with their separator, so a label that is absent (NULL) dropped its separator, e.g. label_join(label_join(vector(1), "a", "", "missing"), "b", "-", "a", "a") lost b="-". Read every source label as coalesce(label, '') like label_replace does.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): skip rows without a sample in timestamp()

timestamp() replaced the input fields with the timestamp before the empty-value filter ran, so a row whose fields were all NULL got a timestamp, e.g. timestamp(-m) for a NULL sample. Filter such rows before the projection, for both the selector and the evaluation-time paths.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 78a7b93292)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): truncate parquet column index min/max for SST writes (#9420)

* fix(mito2): truncate parquet column index min/max for SST writes

The SST writer and the bulk part encoder disabled column index truncation
together with statistics truncation in #6977. Mito never reads the column
index, but every page of a large string or binary column still stored its
full min and max values, uncompressed. Truncate the column index to the
parquet default of 64 bytes and keep chunk statistics untruncated, since
primary key pruning decodes them.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* docs(mito2): note that the column index must not be used to decode primary keys

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit a314ac4281)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: upgrade EC2 runner action to restore runner registration (#9430)

ci: update EC2 runner action for minimum runner version enforcement

Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit 0d502e1fc1)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(catalog): skip backend batch_get on full cache hits (#9431)

fix(catalog): return cached batch values without an empty backend call

Count backend batch_get invocations in regression tests and cover empty input.

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit acebd6b35b)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: make query regression non-blocking for scheduled nightly releases (#9434)

* ci: make query regression non-blocking for scheduled nightly releases

Scheduled nightly releases no longer gate publishing on the query
regression release test. The test still runs (validation policy stays
'all' for automatic releases) and uploads its report, but any result -
failure, timeout, cancellation, or skip - no longer blocks image
publishing or the GitHub release for schedule-triggered runs.

Since reusable-workflow caller jobs cannot use continue-on-error, the
gate is relaxed at the consumers instead: the if conditions of
release-images-to-dockerhub and publish-github-release now also pass
when github.event_name == 'schedule' regardless of the query-regression
result. All other gates (runner allocation, artifact builds,
prepare-release-validation) are unchanged.

Tag-push and manual-dispatch releases remain fully blocking unless a
skip policy is explicitly chosen via the release_validation input.

Signed-off-by: Ning Sun <sunning@greptime.com>

* ci: preserve downstream nightly release jobs after regression failures

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: Ning Sun <sunning@greptime.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Co-authored-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit a9ce6e2d01)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: coalesce equal histogram quantile bucket boundaries (#9451)

* fix: coalesce equal histogram quantile bucket boundaries

Merge numerically equivalent bucket boundaries before monotonicity repair, preserving the allocation-free path for unique boundaries. Share coalescing with histogram_fraction and cover duplicate boundaries with unit and SQL regressions.

Fixes #9443

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: keep duplicate infinity buckets in one histogram group

Confirm classic histogram completion at a series/timestamp boundary or EOF instead of the first positive infinity. Retain lookahead across batches and reject incomplete optimistic groups so equivalent infinity labels reach numeric coalescing together.

Add stream and SQL regressions for duplicate infinity labels, including established bucket counts and EOF. Correct older fixtures to use distinct keys for distinct histogram groups.

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: avoid buffering histogram groups with invalid trailing bounds

Discover candidate bucket counts at the first group boundary and let optimistic validation fall back to safe mode for malformed bounds. Document discovery and cover trailing invalid and null labels with a focused streaming regression.

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit 97913b4b3a)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(sqlness): wait for mirrored inserts before flushing streaming flows (#9424)

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit b78dc4b33b)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: accept a bool value for the CLI --auto-create-schema flag (#9450)

* fix: accept a value for the CLI --auto-create-schema flag

clap derives ArgAction::SetTrue for bool fields, so --auto-create-schema
took no value and could only set true, which is already the default.
Users could not turn automatic schema creation off from the CLI, unlike
metasrv's auto_create_schema = false.

Set action = clap::ArgAction::Set so the flag accepts an explicit
true/false value while keeping true as the default.

Signed-off-by: fengyue-xve <1759519137@qq.com>

* chore: enhance CLI argument for auto_create_schema option

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

---------

Signed-off-by: fengyue-xve <1759519137@qq.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
(cherry picked from commit 329edc88ec)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(otlp): coerce empty and "-" trace attribute strings to null (#9461)

* fix(otlp): coerce empty and "-" trace attribute strings to null

Instrumentations fill numeric attributes from absent headers with an empty
string (PSR-7 getHeaderLine returns ""). Coercing that into an existing
numeric column failed and rejected the whole span.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: trim redundant row from empty-string rewrite test

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit d640ab8bc5)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta-client): bound heartbeat handshake timeout (#9438)

* fix(meta-client): bound heartbeat handshake timeout

Signed-off-by: Aetherance <inkToAether@gmail.com>

* chore(meta-client): fix test license header

Signed-off-by: Aetherance <inkToAether@gmail.com>

* test(meta-client): prefix heartbeat tests with test

Signed-off-by: Aetherance <inkToAether@gmail.com>

---------

Signed-off-by: Aetherance <inkToAether@gmail.com>
(cherry picked from commit 9c69e381ae)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* docs(config): align example configs and config.md with actual defaults (#9460)

* docs(config): align example configs and config.md with actual defaults

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(cmd): compare example defaults inside optional sections

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit e2fd8c5695)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(partition): preserve region mapping during repartition (#9458)

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 90f117b957)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): restore exact filtering for nullable label matchers (#9418)

* fix(promql): restore exact filtering for nullable label matchers

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: verify compound filter semantics and last-row selection

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: consolidate label filter regression coverage

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor: separate filter evaluation and timestamp adaptation

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor: inline predicate and timestamp tests

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor: keep direct IN predicates outside simple filters

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: coerce partition bounds before simple filtering

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: cover partition bound overflow after timestamp widening

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 526dd48bda)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update rust toolchain to 1.98.1 (#9429)

(cherry picked from commit 7f0f592b6b)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: update dev-builder image tag (#9469)

Signed-off-by: greptimedb-ci <greptimedb-ci@greptime.com>
Co-authored-by: greptimedb-ci <greptimedb-ci@greptime.com>
(cherry picked from commit 5affb50bb5)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(metasrv): retry failed heartbeat address updates (#9437)

* fix(metasrv): retry failed heartbeat address updates

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(metasrv): move shared heartbeat helpers to utils

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 65b9a35692)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(operator): bound FlowMirrorTask pending rows to prevent frontend OOM (#9421)

* fix(operator): bound FlowMirrorTask pending rows to prevent frontend OOM

When flownode is slow or unreachable, FlowMirrorTask::detach spawns
unbounded tasks that each pin a full insert batch for ~10s (gRPC
timeout). With continuous writes, in-flight memory grows unboundedly
(throughput x 10s x peers), causing frontend-write OOMKill and a
million-per-hour ERROR storm.

Add a MAX_MIRROR_PENDING_ROWS cap (1M rows) on Inserter: detach
fetch_adds the batch's row count before spawning; if the cap is
exceeded the reservation is rolled back, the batch is dropped, a
DIST_MIRROR_DROPPED_ROW_COUNT metric is bumped, and a warn is logged
instead of spawning. On normal completion the reservation is released
in both success and error paths.

This keeps core writes alive while degrading flow mirror writes to
best-effort, which is the intended trade-off: protect the primary
insert path at the cost of possible flow aggregation gaps.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: address review - balanced reservation, gauge race, log rate limit

P1: Compute pending_rows() from actual per-peer payloads in
self.requests instead of the pre-fanout num_rows field. This fixes
under-reservation for same-table multi-region batches and over-release
in multi-flownode fanout. Each peer task now releases exactly its own
share so reservations and releases balance.

P2: Replace load+set gauge publication with add/sub symmetry.
reserve_mirror_pending_rows does fetch_add + gauge.add, release does
fetch_update + gauge.sub, eliminating the stale-value race.

P3: Rate-limit the dropped-batch warning to at most once per 10s via
MirrorDropLog (atomic compare_exchange), accumulating batch/row counts
between reports.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: fix race in dual-peer test - wait for gauge release before unlocking

The dual-peer test previously only waited for the pending budget to
return to its start value before releasing MIRROR_METRIC_TEST_LOCK.
But release_mirror_pending_rows moves the budget before the gauge, so
a worker's gauge.sub could still be in flight when the next test
acquires the lock and records its baseline, polluting the result.

Now waits for both pending == start AND gauge == gauge_before.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(operator): preserve instant writes and limit mirror failure logs

Reject saturated requests containing instant-TTL rows with the existing retryable write-admission error before mirror or datanode dispatch. Keep normal source-table inserts independent of best-effort mirror drops.

Reuse a smaller process-wide event limiter for both dropped batches and failed peer tasks, reporting at most once per 10 seconds with failure count, peer context, and error chain. Cover actual mirror construction, mixed and instant admission, fast/slow peer shares, failed-task release, and log concurrency.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(operator): reject oversized instant mirror batches permanently

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 198d92407c)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): align subqueries to the absolute evaluation grid (#9400)

* fix(promql): align subqueries to the absolute evaluation grid

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* docs(promql): streamline subquery comments

Remove redundant arithmetic narration and restated test comments. Keep grid, fallback, validation and overflow contracts, and preserve all regression tests and assertions. Regenerate SQLness output only for comment echoes.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 0d2cb1aa7e)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* build(deps): update opensrv-mysql to fix MariaDB 12.3 USE information_schema hang (#9357)

Update opensrv-mysql from f3135e5 to 1f3d88c, which replies a proper EOF packet to COM_FIELD_LIST instead of a malformed OK packet that MariaDB 12.3 misparses and hangs on USE information_schema.

Fixes #9158

Signed-off-by: tian1220A <1573612521@qq.com>
(cherry picked from commit 21536a092b)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): keep vector labels when a computed scalar is on the right (#9478)

* fix(promql): keep vector labels when a computed scalar is on the right

`metric + time()` or `metric * scalar(other)` planned the join with the
scalar side's context, which has no tags, so every series collapsed into
one unlabeled series. Use the vector side's context when the right operand
is scalar-typed.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): update results for labels kept with a computed scalar on the right

v * pi() and v + time() now keep the vector's labels and time index name,
as in Prometheus.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): qualify columns of an unnamed operand in vector joins

A comparison filter's projection has no table name, so after the join its
time index resolved unqualified and was ambiguous when the other operand had
the same column. Also keep the empty-metric branch unchanged for scalar
left operands.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): cover comparison filters joined with a scalar

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 6da5352c53)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(otlp): convert boolean metric attributes into label values (#9468)

Signed-off-by: fengyue-xve <1759519137@qq.com>
(cherry picked from commit 2ebb03fbf6)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf(catalog): optimize single-table lookups in information_schema.partitions (#9473)

* perf(catalog): optimize single-table lookups in information_schema.partitions

Signed-off-by: 风月 <1759519137@qq.com>

* style: apply rustfmt formatting to partitions tests

Signed-off-by: 风月 <1759519137@qq.com>

---------

Signed-off-by: 风月 <1759519137@qq.com>
(cherry picked from commit 4ade13d9b6)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(promql): cover label_replace normalization with distinct series (#9485)

The duplicate-label-set rejection must not fire when a label rewrite keeps
every series distinct through another label. Normalizing mixed histogram
boundary representations (`10.0` -> `10`) while `instance` still differs is
the false positive reported in #9444. The check already accounts for it
since it became a per-timestamp row count instead of a planning-time
rejection, but no fixture covered the positive case. Add one, together
with the aggregation variant that produces a genuine duplicate and must
still fail.

Signed-off-by: fengyue-xve <1759519137@qq.com>
(cherry picked from commit 764ab8bc1a)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): return no point for zero-interval irate/idelta (#9455)

fix(promql): irate/idelta return no point for a zero-ms sample gap

Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>
(cherry picked from commit 0dde3a7b70)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(metasrv): stop parking the heartbeat response on the region supervisor queue (#9474)

* fix(meta-srv): stop parking the heartbeat response on the region supervisor queue

Signed-off-by: lczllx <2181719471@qq.com>

* test(meta-srv): cover the heartbeat response path at the handler level

Signed-off-by: lczllx <2181719471@qq.com>

---------

Signed-off-by: lczllx <2181719471@qq.com>
(cherry picked from commit d1151b0ed6)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): keep metric name for comparison operators (#9332)

* fix(promql): keep metric name for comparison operators

`collect_metric_names` dropped the response `__name__` for every binary
operator except `and` / `or` / `unless`, so comparisons such as
`node_load1 > 5` came back without the metric name. Grafana rendered them
as `{instance="..."}` and alert templates reading `$labels.__name__` got
nothing.

Follow Prometheus' `shouldDropMetricName` / `resultMetric`: arithmetic
operators drop the name, comparisons keep the left-hand name, except when
the comparison returns a bool or is a one-to-one `on(...)` match. A
`group_left` / `group_right` modifier does not reduce the labels, so the
name is kept there.

Both the HTTP PromQL API and the gRPC gateway share
`retrieve_metric_name_and_result_type`, so one fix covers both.

Signed-off-by: Aumnertic <hznuzc@gmail.com>

* docs: document binary_keeps_metric_name

Signed-off-by: Aumnertic <hznuzc@gmail.com>

* fix(promql): handle on(__name__), scalar lhs and group_right

Signed-off-by: Aumnertic <hznuzc@gmail.com>

* chore(promql): correct result-side docs and drop unreachable String arm

Signed-off-by: Aumnertic <hznuzc@gmail.com>

* fix(promql): drop metric name when explicitly ignored

Handle ignoring(__name__) for one-to-one comparisons and cover
comparison, group_left, group_right and unless behavior.

Signed-off-by: Aumnertic <hznuzc@gmail.com>

---------

Signed-off-by: Aumnertic <hznuzc@gmail.com>
(cherry picked from commit c33a6cfa56)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf(promql): reduce timestamp payload allocations and use narrow cursor scans (#9417)

* perf(promql): zero-copy timestamp payload for millisecond zero-offset ranges

RangeManipulate rebuilt the timestamp range payload per sample through
wide-precision i128 conversion even for millisecond input with zero
offset, where the input is already millisecond and nothing changes.
That turned a previously zero-copy path into a per-sample loop plus a
Vec allocation, regressing range-function queries.

Reuse the input timestamp array directly when the time index is already
millisecond and the offset is zero. Other units and non-zero offsets
keep the wide-precision conversion (required so ns/us data near the
native type limits stays correct under large offsets).

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>

* perf(promql): vectorize timestamp payload with plain i64

RangeManipulate rebuilt the timestamp range payload per sample through
wide-precision i128 conversion, turning a hot path into a scalar loop
with a Vec allocation. Realistic timestamps and offsets stay far inside
i64, so the wide path is unnecessary.

- millisecond + zero offset reuses the input array with zero copy;
- other units / offsets shift in one vectorized i64 pass (no i128, no
  per-sample Option, no overflow branch).

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>

* perf(promql): check timestamp payload overflow via arrow try_unary

Replace the silent wrapping shift with try_unary + checked_mul/checked_add.
checked arithmetic lowers to a per-lane overflow flag so the single pass
still vectorizes over the dense i64 buffer; any overflow short-circuits to
an error instead of silently wrapping to a wrong timestamp. No i128, no
per-sample Option.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>

* perf(promql): vectorize cursor scans with i64 fast path

RangeManipulate::calculate_range and InstantManipulate::manipulate
compared every sample through i128 wide-precision arithmetic. For
realistic timestamps and offsets the shifted values stay far inside
i64, so the wide path was pure overhead.

Split both scans into an i64 fast path (checked arithmetic precomputed
once per batch) and an i128 exact fallback. Any input that would
overflow i64 falls back to the i128 scan, keeping window membership
semantics identical.

Verified: 47/47 range_manipulate + instant_manipulate tests, 275/275
promql crate tests, 1612 randomized cases byte-identical to i128 scan.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>

* refactor(promql): generalize cursor scans and share endpoint precheck

Address review comments on PR #9417:
- Generalize scan_ranges/scan_instants over Nanoseconds (i64/i128),
  eliminating the duplicated scan bodies.
- Extract shared narrow_scan precheck used by both RangeManipulate and
  InstantManipulate, with identical checked endpoint logic.
- Trim doc comments to one line; fix factual errors about fallback
  trigger (2262 for ms/s, not native type limits).

Semantics unchanged: same cursor algorithm, same overflow fallback,
same i64-vs-i128 selection, same stale/null/tie handling.

Verified: 47/47 range_manipulate + instant_manipulate tests, clippy clean.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>

* perf(promql): vectorize payload with try_unary and add scan carrier bench

Replace the per-sample i128 payload loop with a single arrow try_unary
pass: i128 intermediate holds any native timestamp shift, only the final
millisecond result must fit i64. This is semantically equivalent to the
original i128 implementation (no silent wrap, no per-sample branching).

Add cursor_scan_carriers A/B benchmark: i64 fast path vs i128 exact
fallback on four workloads. Add five deterministic differential tests
pinning scan_ranges::<i64> and scan_ranges::<i128> to be pointwise
equivalent on boundary shapes (sub-ms samples, fallback triggers,
nanosecond extremes, seeded precision matrix).

Bench (N=4096, 15s cadence):
- payload wall_time 20x15s: 74.94us -> 48.98us (-35%)
- payload wall_time 240x15s: 102.44us
- scan i64 fast path: 10.97us vs i128 19.84us (1.81x)

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* perf(promql): restore zero-copy payload for ms+offset=0 and fix comment

Restore the zero-copy fast path for millisecond timestamps with zero
offset, guarded by the input column's exact data type (Timestamp
(Millisecond, None)) rather than self.time_unit. The old time_unit guard
would accept a tz-carrying millisecond column and produce a dictionary
value type disagreeing with the output schema; the datatype guard is the
correct check.

Also fix the comment: arrow's try_unary is a fallible kernel that LLVM
cannot auto-vectorize; the win over the hand-written loop comes from
eliminating the per-sample Option/Vec and branches, not SIMD.

Add a regression test with a timezone-carrying millisecond index to
prove it falls back to the try_unary path.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* perf(promql): replace try_unary with unary + Cell overflow flag

Arrow's try_unary is a fallible kernel that LLVM cannot auto-vectorize.
Replace it with the infallible unary kernel: the closure is a plain
i64 -> i64 mapping that sets a Cell<bool> on overflow instead of
returning Err per sample. The flag is checked once after the loop.

This removes the per-lane Result construction and lets LLVM vectorize
the shift+truncate sequence. Bench (N=4096, 15s cadence):
- window4x15s: 42.12us -> 28.77us (-32%)
- window20x15s: 48.98us -> 27.68us (-43%)
- window240x15s: 102.44us -> 27.61us (-73%)
- regular_sparse: 90.04us -> 29.20us (-68%)

All cases now run at ~27-29us regardless of window size; the zero-copy
path dominates for ms+offset=0.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* perf(promql): dispatch payload conversion on null_count; drop wall-time bench

Null-free batches (dominant PromQL case) take arrow's vectorizable unary
kernel with a Cell<bool> overflow flag; nullable batches take try_unary so
a NULL slot's arbitrary underlying value cannot raise a false batch-wide
overflow error. Add a regression test exercising both paths.

Remove bench_range_manipulate_wall_time (all-ms zero-offset cases only
exercised the zero-copy path, measuring nothing about the conversion) and
its orphaned helpers; restore the cursor-scan bench helpers it shared.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(promql): keep nullable timestamp cursor scans on i128

NULL slots may contain unordered arbitrary values, so the endpoint-only
narrow-scan proof cannot bound their interior timestamp arithmetic. Both
range and instant scans now attempt i64 only for null-free timestamp arrays
and otherwise retain the existing exact i128 path.

Add focused interior-NULL regressions through both active entrypoints,
checking concrete selections and output against the existing wide scan.
The null-free fast path and timestamp payload conversion are unchanged.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>
(cherry picked from commit b3c756a7dd)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(metasrv): stop holding the pusher registry lock across mailbox sends (#9445)

fix(meta-srv): stop holding the pusher registry lock across mailbox sends

Signed-off-by: lczllx <2181719471@qq.com>
(cherry picked from commit 7605f27608)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito): flush memtables containing skip-WAL writes on region close (#9440)

* fix(mito): flush unlogged writes when closing regions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito): align skip-WAL naming and metric recovery tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit a026b3b280)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: enhance benchmark workflows with version comparisons and diagnostics (#9383)

* ci: update Agent and Traces benchmark runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use unrestricted benchmark runtimes by default

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: retry benchmark image pulls before generating datasets

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: increase tracesbench defaults to 8 CPU and 32 GiB

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: configure tracesbench query timeout and update runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: update tracesbench runtime and configure import scan workers

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use shared parallel tracesbench import preparation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: update tracesbench runtime to preserve Tempo trace requests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: summarize tracesbench query results and lifecycle

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: show benchmark resource usage and update parallel loader runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(ci): resolve shared summary module in existing test harness

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: select tracesbench engines and GreptimeDB image version

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: update benchmark runtime with parallel Agent preparation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: select benchmark runtime registry by cloud provider

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: retry long-range benchmark image pulls

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: compare baseline and candidate GreptimeDB benchmarks

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: pin published dual-version benchmark runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: prepare agent benchmark indexes with updated runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: validate agent admission and render traces version comparison

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use runtime with traces index preparation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use consolidated runtime for benchmark version reports

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add license header to benchmark comparison tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(ci): read legacy Tempo ingestion metrics in traces summaries

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: require tracesbench query plan artifacts

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: use optimized tracesbench report runtime

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: update tracesbench runtime to merged report validation

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 6e1a137411)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): accept evaluation times before the Unix epoch (#9480)

* fix(promql): accept evaluation times before the Unix epoch

Prometheus accepts negative timestamps in query parameters; GreptimeDB
rejected them while parsing and would panic converting them in the planner.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: update flow_tql error for a negative TQL end time

A negative end now parses and fails the end >= start check instead of
the timestamp conversion.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): reject evaluation times outside the millisecond range

The signed millisecond conversion cast u128 to i64 unchecked, so a far-past
time wrapped into the future. Reuse the checked conversion from
label_values and return an error instead.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: drop an unused import

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): round query times to milliseconds like Prometheus

Numeric times round the fraction of a second to milliseconds, and
pre-epoch times floor sub-millisecond parts, matching Prometheus'
parseTime and timestamp.FromTime.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): round the float fraction before converting to nanoseconds

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: skip the overflow case where SystemTime cannot hold it

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit a8e293fd79)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: unify unknown flow option rejection across OSS and enterprise builds (#9319)

* fix(operator): unify unknown flow option rejection across OSS and enterprise builds

Remove the enterprise-gated passthrough that let unknown flow options
through SQL-layer validation. Both OSS and enterprise binaries now reject
unknown flow options with a single 1004(InvalidArguments) error at the
SQL layer, producing identical sqlness output.

The metasrv-side validate_flow_options remains as a defensive check.

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): use DISTINCT output columns as sink key for streaming flows

When a streaming flow's source table has no primary key, sink key
inference produced an empty key. For SELECT DISTINCT v, the DISTINCT
establishes v as the output key, but the inference only looked at
source primary keys. Combined with the zero-valued placeholder
timestamp, all output rows shared the same storage identity and were
deduplicated into a single row.

Use all non-time-index output columns as the sink key when the plan
is a DISTINCT and no source primary key was inherited.

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): preserve extension options and complete DISTINCT sink identity

Keep enterprise CREATE FLOW options available to the existing handler and report unknown ordinary options as InvalidArguments with the OSS SQL error text. Infer DISTINCT sink keys from every selected non-time-index output column, including when source key columns survive. Wait for asynchronous mirrors in show_create_flow and exercise 1000 distinct sink rows in the SQL regression.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(flow): align meta unknown option status assertions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(flow): use valid aliases in DISTINCT sink regression SQL

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(flow): record complete DISTINCT sink SQL results

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 178a0021b8)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): return range-vector results for instant queries (#9477)

* fix(promql): return range-vector results for instant queries

An instant query of a range selector or subquery (e.g. `up[5m]`) returned
the internal range arrays, which panicked when the frontend decoded them as
dictionaries. Plan such queries as the selected samples with their own
timestamps, and reject range-vector expressions in range queries like
Prometheus does.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): keep encode_substrait on a valid range query

A range query of a range-vector expression now fails like in Prometheus.
Wrap the subquery in max_over_time so the case still encodes both
RangeManipulate nodes through substrait.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): plan wrapped range-vector roots and reject a zero subquery step

TQL EXPLAIN, ANALYZE and AS alias wrap the root in an extension, which the
range-vector path did not unwrap. A zero subquery step divided by zero.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): cover wrapped range-vector roots and a zero subquery step

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): default subquery step and reject range vectors in range queries

A root subquery without a step uses the one-minute default instead of
the query step. The HTTP range query endpoint rejects range-vector
expressions even when start equals end.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: cover root subquery default step

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor(promql): align root subqueries with subquery_child_window

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): return millisecond timestamps for range-vector results

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor(promql): keep the time index timezone and name its table in errors

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit b913be2af7)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(query): stop reporting region scans as hash partitioned (#9491)

* fix(query): stop reporting region scans as hash partitioned

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): use current DataFusion plan downcasts

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): match null rendering in join regression

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): refresh union plan for unknown scan partitioning

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): refresh shared plans after distribution rule removal

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): cover semi joins with remote dynamic filtering

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): strengthen partitioned merge scan regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): remove unused merge scan test helpers

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(perf): measure merge scan hash repartition cost

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(query): adapt merge scan deserialization to two-arg constructor

Adapt the production and test serializer callers added by #9206 and remove the obsolete partition_cols assertion.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(query): adapt nested merge scan integration constructor

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 32e8467cda)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(table): require complete primary keys for hash partitioning (#9507)

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 28ec3ff5aa)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(deps): bump OpenDAL to 0.59.4 for OSS RRSA credentials (#9520)

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit e6f3ca10a5)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): default a stepless subquery to a one-minute step (#9481)

* fix(promql): default a stepless subquery to a one-minute step

`x[5m:]` evaluated the inner expression at the query step, which is 1s
for instant queries. Prometheus uses the evaluation interval, one minute
by default.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): cover the default subquery step

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): cover the default subquery step in a range query

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 845fc0af9c)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): follow IEEE 754 float semantics in comparisons and aggregations (#9479)

* fix(promql): follow IEEE 754 float semantics in operators and aggregations

- Comparisons: Arrow orders floats totally (NaN == NaN, NaN > +Inf); PromQL
  comparisons with NaN are false except !=.
- min/max: skip NaN unless the whole group is NaN. DataFusion's grouped
  min/max let NaN replace and be replaced by neighbours, so the result
  depended on input order.
- topk/bottomk rank NaN last.
- round() rounds ties up and returns NaN for a zero step, like Prometheus.
- sqrt() of a negative number is NaN instead of an error.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): round() without a step rounds to integers

The planner passed a zero step for single-argument round(), which now
yields NaN like Prometheus; pass 1 instead. Found by the compatibility harness.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): apply the sqrt guard to builtin functions and register min/max for plan decoding

sqrt resolves through the session's builtin functions, not the UDF branch.
Distributed plans are decoded with the serializer's session, which did not
know prom_min/prom_max.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): cover NaN and rounding semantics

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): keep the nested count rewrite for PromQL min/max

count(min(x) by (l)) was rewritten by count_nest_aggr only for the builtin
min/max names. prom_min/prom_max return NULL for the same groups, so the
rewrite stays valid. Update plan snapshots for the IEEE comparison, sqrt,
topk and min/max changes.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): register prom_min/prom_max in the function registry

Distributed plans split an aggregation into state and merge steps only for
functions whose state function exists in FUNCTION_REGISTRY; prom_min and
prom_max lived in the promql crate, so min/max stopped being pushed down to
datanodes. Move them to common-function and register them before the state
functions are derived.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: update sqlness results for PromQL float semantics

Plans and column names now show prom_min/prom_max, the NaN guards of
comparisons and sqrt, and the NaN sort key of topk. round() results follow
Prometheus' formula. Distributed min/max keep the state/merge split.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): keep column names and NULLs in IEEE comparisons and sqrt

Rewriting comparisons as `a op b AND NOT (isnan(a) OR isnan(b))` changed
TQL output column names and turned NULL operands into true for `!=` and
false for the bool forms. Implement comparisons and sqrt as PromQL UDFs
that propagate NULL and report the column name of the expression they
replace; prom_min/prom_max report min/max column names the same way.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: restore column names in sqlness results for PromQL float semantics

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): share the prom_min/prom_max groups accumulator with tests

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): keep distinct names for prom_min/prom_max

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit f3a0841f4f)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(query): keep filter fetch when rewriting matches term (#9521)

* test: cover matches term limit dropped by constant rewrite

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(query): keep filter fetch when rewriting matches term

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 483d1e49a3)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: adapt release OptionMap iteration to Rust 1.98

Signed-off-by: WenyXu <wenymedia@gmail.com>

* build: reduce checkout-specific inputs for cache reuse (#9198)

* test(mito2): cover regex inverted index pruning

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* build: keep build inputs checkout-relative for cross-worktree cache reuse

Two small changes so compilations are byte-identical across worktrees
of the same commit and can share a build cache (mbx, sccache):

- src/common/version/build.rs: derive the workspace root relative to
  CARGO_MANIFEST_DIR instead of the CARGO_WORKSPACE_DIR injected by
  .cargo/config.toml (an absolute path). The env var remains as an
  explicit override for the submodule case; the [env] injection in
  .cargo/config.toml is removed so the relative derivation is used by
  default.
- Replace three env!("CARGO_MANIFEST_DIR") uses in test/bench code
  (servers prom_decode bench + decode test, common-meta test_util)
  with a runtime std::env::var lookup, so the compiled artifact no
  longer embeds the checkout path.

No behavior change: same files are read, same version info emitted.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* build: address cross-worktree cache review feedback

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit f50f42140a)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito): preserve version ordering in bulk merges and flushes (#9517)

* fix(mito): preserve delete barriers in flat last-non-null dedup

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito): respect sequence ranges in bulk merges and flushes

Signed-off-by: evenyag <realevenyag@gmail.com>

* perf(mito): avoid merge selection on the bulk write path

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit 674ba72669)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: benchmark the object store WAL against S3 from an EC2 runner (#9536)

* ci: benchmark the object store WAL against S3 from an EC2 runner

Add a workflow_dispatch workflow that loads the o11ybench agent observability
dataset into a fresh GreptimeDB container per cell and compares three WAL
configurations that differ only in [wal]: Raft Engine with sync_write on the
instance store, and the object store WAL in durable and enqueued mode, with
the data files on S3 in the runner's region for all three. Rounds alternate
the configuration order. A summary reports rows per second, D/R and E/R as
the geometric mean over rounds, append acknowledgement p50 and p99, and the
object store WAL counters. A smoke mode runs the same chain on ubuntu-latest
against the MinIO fixture.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* ci: scrape WAL metrics on failed loads and keep logic out of the benchmark YAML

Scrape the object store WAL metrics before stopping a cell's server whether
the load succeeded or failed, so failed cells keep their stall, poison and
create failure counters. Build the summary from the plan so that cells which
never ran are listed and leave their ratios undefined. Move budget, region,
runtime image, manifest and credential scan logic from the workflow into the
helper scripts, and drive the cell lifecycle in the tests through fake docker
and curl binaries.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* ci: drop redundant required flags from benchmark inputs

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* ci: fail a benchmark cell whose metrics scrape fails and zero untouched WAL counters

A load that succeeds without its metrics scrape now fails the cell with exit
code 4 and records metrics_scraped in cell.json. The summary reports an
event counter that never registered as zero when the cell is an object store
WAL cell whose scrape succeeded and created objects, and as unknown otherwise.
Planned cells that never ran are read through the same path as recorded ones.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* ci: register the S3 benchmark workflow for dispatch from its branch

A workflow_dispatch workflow can only be dispatched once GitHub has
registered it, which happens on the default branch or when one of its
triggers fires. A push trigger limited to this branch registers it; validate
runs only for workflow_dispatch, so a push run skips every job.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* ci: delete the benchmark S3 prefix one object at a time

boto3 now sends a CRC checksum instead of Content-MD5 with DeleteObjects,
which the MinIO fixture of the smoke run rejects. DeleteObject needs neither
and the prefix of one run holds at most a few thousand objects.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* ci: default the S3 benchmark to c6id.2xlarge

c7id is not offered in the runner region ap-southeast-1, where DescribeInstanceTypes
rejects it. c6id is the compute optimized x86 family with a local NVMe instance
store that the region offers, with the same 8 vCPU and 16 GiB at 2xlarge.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* ci: drop the registration trigger of the S3 benchmark workflow

The workflow is registered now, so dispatches no longer need a push trigger
on its branch.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* ci: fail an S3 benchmark run on EC2 without a local instance store

Without an instance store the Raft Engine WAL of the R cells silently stayed
on the root EBS volume, so the run measured a different baseline. Only a smoke
run may keep it on the root volume now.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* ci: let the instance store check fail its workflow step

The step piped wal-disk into tee without pipefail, so a rejected host still
passed. wal-disk now writes its record itself and the step runs it alone.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit c1a0ca9595)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): ignore missing inverted indexes in old SSTs (#9545)

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 287ced2de2)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: prevent stale leader metadata after concurrent writes (#9549)

fix: fence leader cache fills around store mutations

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit 394562aa9e)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): stop fabricating histogram fold hash partitioning (#9509)

* fix(promql): stop fabricating histogram fold hash partitioning

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): cover histogram fraction binary join missing rows

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): normalize dictionary labels in histogram regression

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: cover histogram joins on logical metric tables

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: specify insert columns for logical metric fixture

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: use supported histogram child replacement API

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): keep histogram fold fixtures truthful

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 2039f508fc)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(client): support region request reuse and route refresh hints (#9540)

* feat(client): support shared region requests without deep cloning

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: propagate route refresh retry hints

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style: preserve local error variant imports in retry hints

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(mito2): distinguish staging rejection from write backpressure

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): preserve legacy retry hints in instruction replies

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(client): inline retry hint compatibility cases

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test(datanode): align batch-open retry hint assertion

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style: use imported types in region request tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 684c8ff291)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(operator): centralize write partitioning and dispatch (#9541)

* refactor(operator): centralize write partitioning and dispatch

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(operator): preserve write dispatch behavior during refactor

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(operator): import write dispatch error paths

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(operator): align dispatch imports and test utilities

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(operator): use imported types in partition tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit a2d28e6060)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(operator): retry failed writes with refreshed partitions (#9542)

* feat(operator): retry failed writes with refreshed partitions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(operator): simplify retry test type references

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(operator): clarify write request rebuilding names

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(operator): encapsulate write retry attempts

Signed-off-by: WenyXu <wenymedia@gmail.com>

* style(operator): group inserter configuration methods

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(operator): share routed peers and separate retry decisions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor(operator): avoid temporary write routing allocations

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit d86741d0ad)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(frontend): retry transient distributed write failures (#9543)

* refactor(operator): route shared requests through NodeManager

Remove the concrete NodeClients requester API and use the existing node manager for owned and shared requests. Preserve the original write hint while the legacy error adapter remains in place.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(meta): preserve retry policy through error hints

Propagate client errors through External and let DDL read retry hints while preserving cleanup and terminal-child policies. Region transport failures retain the legacy Retryable classification, which now also participates in bounded write retries.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(operator): restore sparse partition tags when rebuilding writes

Decode retained primary keys when refreshed partition expressions need different tags. Keep the partitioner and multi-region sparse write regression tests with the fix.

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(operator): stop detached write retries when callers cancel

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat(frontend): configure retries across distributed write paths

Wire the shared dispatcher through frontend and logical batching, preserving partition columns and request headers. Keep standalone retries disabled and include configuration, batching, and storage regression coverage.

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 863eebdecb)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(table): stop claiming tag order for logical metric scans (#9522)

* test: cover metric logical scan tag ordering

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(table): only claim projected primary key prefix ordering

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(table): keep physical key ordering for metric projections

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(table): compare unknown partition counts explicitly

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(table): model scoped internal TSID scan properties

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 75bc4ce46d)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(promql): stop claiming scalar calculate preserves order (#9523)

* test: cover scalar calculate order contract

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(promql): stop claiming scalar calculate preserves order

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* docs(test): describe scalar ordering regression as pre-fix behavior

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 0e40281262)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: align fulltext bloom tokens with matches_term and use them for LIKE (#9470)

* fix: align fulltext bloom tokens with matches_term and use them for LIKE

The bloom fulltext index stored tokens that did not follow
matches_term's word boundaries (`_`, ASCII next to Han, Han substrings),
so pruning dropped rows matches_term accepts.

Tokens and query probes now share the character classes of
matches_term, Han runs are indexed as bigrams, and LIKE patterns use the
same probes. The bloom blob is written under a new key; blobs from older
versions are no longer probed. Tantivy no longer serves matches_term.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: drop unused lazy_static from index and avoid typo-flagged test tokens

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* docs: link like_probes to tokenize instead of the downstream ScriptTokenizer

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* refactor: name probe extraction with verbs and document CharClass

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* feat: rebuild legacy fulltext bloom indexes and probe lower(col)

The manifest records the fulltext bloom version per column, so ADMIN
BUILD_INDEX now rebuilds SSTs whose bloom index predates the current
tokens, and ssts_index_meta shows the version.

Case-insensitive bloom indexes also store the tokens of the whole
lowercased text for rows where it can differ (U+0130, U+212A, sigma),
so matches_term(lower(col), ..) and lower(col) LIKE .. use the index.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test: cover legacy bloom index rewrite by compaction in the compat case

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: log outdated fulltext bloom indexes in the consistency check

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* docs: explain the unrecorded fulltext bloom version

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 5c4d9eb4f8)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(query): guard deep expressions during query planning (#9564)

* fix(query): avoid speculative clones in constant normalization

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(query): reject deeply nested expressions before optimization

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit 00aa65de7a)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(operator): reuse catalog table resolution for write retries (#9574)

* fix(operator): inherit logical table partitions when retrying writes

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(operator): resolve retried writes through catalog manager

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 36423a95c6)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(query): accept narrowed nullability in MergeScan remote schema check (#9592)

On append-mode tables a region reports an exact row count. When it is
zero, DataFusion's AggregateStatistics replaces the region-side
__min_state/__max_state aggregate with a non-null struct literal, so the
remote stream advertises the state column as non-nullable. The MergeScan
schema check required equal nullability and failed the query.

Accept a remote field that is non-nullable where the expected field is
nullable, matching DataFusion's own physical-plan schema invariant.
Batches are already rebuilt with the expected schema. Widening is still
rejected, and nested field nullability is still compared exactly as part
of the data type.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit ef1298ce02)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(table): stop claiming exact null and distinct counts for append scans (#9593)

RegionScanExec reported null_count = 0 and distinct_count = total_rows for
every column of an append-only table. DataFusion answers count(col) and
count(DISTINCT col) from these statistics, so both returned the row count.
Keep the exact row count and an exact zero null count for non-nullable
columns, so count(*) is still answered from statistics.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 5cfdc30a69)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: avoid pinning memtables during series-index reconciliation (#9588)

Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit ce66d8d4ac)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* perf(query): infer safe aggregate input bounds from HAVING (#9582)

* perf(query): infer safe aggregate input bounds from HAVING

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(query): refresh explain snapshots for aggregate bound inference

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(query): avoid reinferring bounds across MergeScan

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(query): validate aggregate bound column provenance

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(query): retain zero-match groups for nonpositive HAVING thresholds

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(query): assert inferred bounds in remote scan plans

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
(cherry picked from commit af396e60a3)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(query): keep state column names unique when splitting aggregates (#9594)

* fix(query): keep state column names unique when splitting aggregates

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(query): probe allocated names when renaming colliding state columns

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit f7a3a7033c)
Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: bump version to 1.3.0-beta.2

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: greptimedb-ci <greptimedb-ci@greptime.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: Aarav <aaravsjadav@gmail.com>
Signed-off-by: Ning Sun <sunning@greptime.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: fengyue-xve <1759519137@qq.com>
Signed-off-by: Aetherance <inkToAether@gmail.com>
Signed-off-by: tian1220A <1573612521@qq.com>
Signed-off-by: 风月 <1759519137@qq.com>
Signed-off-by: Niladri Adhikary <niladrix719@gmail.com>
Signed-off-by: lczllx <2181719471@qq.com>
Signed-off-by: Aumnertic <hznuzc@gmail.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
Co-authored-by: greptimedb-ci <greptimedb-ci@greptime.com>
Co-authored-by: DeviousCardi <115358213+DeviousCardi@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: fengyue-xve <1759519137@qq.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Aether <98370028+Aetherance@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: zhaiAohan <1573612521@qq.com>
Co-authored-by: Niladri Adhikary <91966855+niladrix719@users.noreply.github.com>
Co-authored-by: Dragon Roar <2181719471@qq.com>
Co-authored-by: Aum <hznuzc@gmail.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
2026-10-15 20:34:19 +08:00

125 KiB
Raw Blame History

Configurations

Standalone Mode

Key Type Default Descriptions
default_timezone String Unset The default timezone of the server.
default_column_prefix String Unset The default column prefix for auto-created time index, value, and native histogram columns.
Legacy OTLP summary columns keep their historical greptime_ prefix.
auto_create_table Bool true Server-side global switch for auto table creation on write.
When false, a missing table is never auto-created even if the request sets the auto_create_table hint to true. Default: true.
experimental_metric_export Bool false Enables experimental Parquet COPY DATABASE using shared Metric physical scans.
Resume requires the previous export and storage writes to have ended; HTTP timeout is not confirmation.
user_provider String Unset The user provider for authentication.
Examples: "static_user_provider:file:/path/to/users", "static_user_provider:cmd:greptime_user=greptime_pwd"
Password verifier formats: "plain:", "pbkdf2_sha256::<hex_salt>:<hex_hash>",
"mysql_native_password:<hex_sha1_sha1_password>",
"pg_scram_sha256::<hex_salt>:<hex_stored_key>:<hex_server_key>"
"pbkdf2_sha256" and "pg_scram_sha256" protect passwords at rest, but such users cannot log in over MySQL:
static_user_provider and watch_file_user_provider only negotiate mysql_native_password, which cannot verify these formats.
"mysql_native_password" is MySQL-specific and cannot authenticate over PostgreSQL at all.
PostgreSQL SCRAM only covers "plain" and "pg_scram_sha256" users; if any user is "pbkdf2_sha256" or
"mysql_native_password", PostgreSQL falls back to cleartext password auth for every user.
For "pg_scram_sha256" users, keep the default iteration count (4096) and salt length (16): both are
observable in the SCRAM server-first message, and non-default values weaken resistance to username
enumeration.
max_in_flight_write_bytes String 0 Maximum total memory for all concurrent write request bodies and messages (HTTP, gRPC, Flight).
Set to 0 to disable the limit. Default: "0" (unlimited)
write_bytes_exhausted_policy String wait Policy when write bytes quota is exhausted.
Options: "wait" (default, 10s timeout), "wait()" (e.g., "wait(30s)"), "fail"
init_regions_in_background Bool false Initialize all regions in the background during the startup.
By default, it provides services after all regions have been initialized.
init_regions_parallelism Integer 16 Parallelism of initializing regions.
enable_telemetry Bool true Enable telemetry to collect anonymous usage data. Enabled by default.
runtime -- -- The runtime options.
runtime.global_rt_size Integer Auto The number of threads to execute the runtime for global read operations.
Defaults to max(num_cpus, 2).
runtime.compact_rt_size Integer Auto The number of threads to execute compact operations.
Defaults to max(num_cpus / 2, 2).
runtime.compact_rt_max_blocking_threads Integer Auto The maximum number of blocking threads for compact operations.
Defaults to max(num_cpus / 2, 2).
runtime.experimental_workload_scheduler -- -- Experimental weighted, work-conserving query/write task scheduler.
runtime.experimental_workload_scheduler.enable Bool false Enable when concurrent queries and writes interfere with each other—for example, when long-running queries increase ingestion latency.
The weights set their relative runtime shares while both are backlogged. Disabled by default.
runtime.experimental_workload_scheduler.query_weight Integer 2 Relative query share while both query and write workloads are backlogged.
runtime.experimental_workload_scheduler.write_weight Integer 8 Relative write share while both query and write workloads are backlogged.
runtime.experimental_workload_scheduler.sample_every_polls Integer 16 Number of polls between scheduler fairness samples. Must be greater than zero.
http -- -- The HTTP server options.
http.addr String 127.0.0.1:4000 The address to bind the HTTP server.
http.timeout String 0s HTTP request timeout. Set to 0 to disable timeout.
When synchronous Prometheus, OTLP metrics, or ordinary-table batching is enabled, a nonzero timeout is
raised to at least the largest active flush interval plus 1 second. The intervals come from
pending_rows_batcher.logical_table.pending_rows_flush_interval and pending_rows_batcher.pending_rows_flush_interval.
http.body_limit String 64MB HTTP request body limit.
The following units are supported: B, KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB.
Set to 0 to disable limit.
http.enable_cors Bool true HTTP CORS support, it's turned on by default
This allows browser to access http APIs without CORS restrictions
http.cors_allowed_origins Array Unset Customize allowed origins for HTTP CORS.
http.enable_api_server Bool false Whether to start the dedicated public HTTP API server. This server serves
only the v1 interfaces plus the dashboard, and shares every other [http]
option with the main server. It is disabled by default; set to true to enable.
http.api_server_addr String 127.0.0.1:4006 The address to bind the dedicated HTTP API server, in the same form as addr.
Defaults to 127.0.0.1:4006.
grpc -- -- The gRPC server options.
grpc.bind_addr String 127.0.0.1:4001 The address to bind the gRPC server.
grpc.runtime_size Integer 8 The number of server worker threads.
grpc.max_connection_age String Unset The maximum connection age for gRPC connection.
The value can be a human-readable time string. For example: 10m for ten minutes or 1h for one hour.
Refer to https://grpc.io/docs/guides/keepalive/ for more details.
grpc.tls -- -- gRPC server TLS options, see mysql.tls section.
grpc.tls.mode String disable TLS mode.
grpc.tls.cert_path String Unset Certificate file path.
grpc.tls.key_path String Unset Private key file path.
grpc.tls.watch Bool false Watch for Certificate and key file change and auto reload.
For now, gRPC tls config does not support auto reload.
mysql -- -- MySQL server options.
mysql.enable Bool true Whether to enable.
mysql.addr String 127.0.0.1:4002 The addr to bind the MySQL server.
mysql.runtime_size Integer 2 The number of server worker threads.
mysql.keep_alive String 0s Server-side keep-alive time.
Set to 0 (default) to disable.
mysql.prepared_stmt_cache_size Integer 10000 Maximum entries in the MySQL prepared statement cache; default is 10,000.
mysql.tls -- -- --
mysql.tls.mode String disable TLS mode, refer to https://www.postgresql.org/docs/current/libpq-ssl.html
- disable (default value)
- prefer
- require
- verify-ca
- verify-full
mysql.tls.cert_path String Unset Certificate file path.
mysql.tls.key_path String Unset Private key file path.
mysql.tls.watch Bool false Watch for Certificate and key file change and auto reload
postgres -- -- PostgresSQL server options.
postgres.enable Bool true Whether to enable
postgres.addr String 127.0.0.1:4003 The addr to bind the PostgresSQL server.
postgres.runtime_size Integer 2 The number of server worker threads.
postgres.keep_alive String 0s Server-side keep-alive time.
Set to 0 (default) to disable.
postgres.tls -- -- PostgresSQL server TLS options, see mysql.tls section.
postgres.tls.mode String disable TLS mode.
postgres.tls.cert_path String Unset Certificate file path.
postgres.tls.key_path String Unset Private key file path.
postgres.tls.watch Bool false Watch for Certificate and key file change and auto reload
opentsdb -- -- OpenTSDB protocol options.
opentsdb.enable Bool true Whether to enable OpenTSDB put in HTTP API.
influxdb -- -- InfluxDB protocol options.
influxdb.enable Bool true Whether to enable InfluxDB protocol in HTTP API.
influxdb.default_merge_mode String last_non_null Default merge mode for tables automatically created by InfluxDB protocol.
Available values: "last_non_null", "last_row".
pending_rows_batcher -- -- Ordinary-table batching for opted-in ingestion protocols.
PENDING_ROWS_BATCH_SYNC defaults to true for both batchers. Set it to false to acknowledge
queue admission without waiting for storage; later failures cannot be returned to the client.
Omitted or empty protocols disables batching. Prom without metric engine uses this batcher.
OTLP logs, traces and ordinary metrics use this batcher.
MySQL and PostgreSQL use the same acknowledgement policy. Protocol timeouts are unchanged.
Single-connection writes and INSERT SELECT may incur additional flush waits.
pending_rows_batcher.pending_rows_flush_interval String 0s Flush interval measured from the first pending submission. Zero disables batching.
pending_rows_batcher.max_batch_rows Integer 100000 Flush after a complete submission reaches this row threshold.
pending_rows_batcher.max_concurrent_flushes Integer 256 Maximum concurrent flushes shared by the frontend batcher.
pending_rows_batcher.worker_channel_capacity Integer 65536 Maximum queued submissions per table worker.
pending_rows_batcher.max_inflight_requests Integer 3000 Maximum admitted original requests awaiting completion.
pending_rows_batcher.flow_notification_queue_capacity Integer 1024 Maximum number of queued table Flow notifications.
pending_rows_batcher.logical_table -- -- Metric-engine logical-table batching for Prom remote write and non-legacy OTLP metrics.
Requires prom_store.with_metric_engine. Logs, traces and legacy metrics are not eligible.
Enable independently with protocols and a nonzero flush interval.
Omitted fields use independent defaults, not parent settings.
Empty protocols or a zero interval disables logical batching without fallback.
Omitting this entire section preserves legacy Prom batching; it does not enable OTLP batching.
pending_rows_batcher.logical_table.pending_rows_flush_interval String 0s Flush interval measured from the first pending submission. Zero disables batching.
pending_rows_batcher.logical_table.max_batch_rows Integer 100000 Flush after a complete submission reaches this row threshold.
pending_rows_batcher.logical_table.max_concurrent_flushes Integer 256 Maximum concurrent flushes shared by Prom and OTLP metrics.
pending_rows_batcher.logical_table.worker_channel_capacity Integer 65536 Maximum queued submissions per physical-table worker.
pending_rows_batcher.logical_table.max_inflight_requests Integer 3000 Maximum admitted original requests awaiting completion.
pending_rows_batcher.logical_table.flow_notification_queue_capacity Integer 1024 Maximum number of queued logical-table Flow notifications.
jaeger -- -- Jaeger protocol options.
jaeger.enable Bool true Whether to enable Jaeger protocol in HTTP API.
otlp -- -- OpenTelemetry protocol options.
otlp.enable Bool true Whether to enable OpenTelemetry protocol in HTTP API.
otlp.trace_ingest_chunk_size Integer 512 Maximum spans per trace ingest chunk. Set to 0 to disable splitting.
otlp.experimental_enable_resource_info Bool false Whether to synthesize the greptime_otel_resource_info table from OTLP metric
resource attributes, so metrics-only services reach the semantic graph.
prom_store -- -- Prometheus remote storage options
prom_store.enable Bool true Whether to enable Prometheus remote write and read in HTTP API.
prom_store.with_metric_engine Bool true Whether to store the data from Prometheus remote write in metric engine.
prom_store.prom_validation_mode String strict Whether to enable validation for Prometheus remote write requests.
Available options:
- strict: deny invalid UTF-8 strings (default).
- lossy: allow invalid UTF-8 strings, replace invalid characters with REPLACEMENT_CHARACTER(U+FFFD).
- unchecked: do not valid strings.
wal -- -- The WAL options.
wal.provider String raft_engine The provider of the WAL.
- raft_engine: the wal is stored in the local file system by raft-engine.
- kafka: it's remote wal that data is stored in Kafka.
- experimental_object_store: the wal is stored as objects in an object store.
Notes: experimental and not supported yet.
wal.dir String Unset The directory to store the WAL files.
It's only used when the provider is raft_engine.
wal.file_size String 128MB The size of the WAL segment file.
It's only used when the provider is raft_engine.
wal.purge_threshold String 1GB The threshold of the WAL size to trigger a purge.
It's only used when the provider is raft_engine.
wal.purge_interval String 1m The interval to trigger a purge.
It's only used when the provider is raft_engine.
wal.read_batch_size Integer 128 The read batch size.
It's only used when the provider is raft_engine.
wal.sync_write Bool false Whether to use sync write.
It's only used when the provider is raft_engine.
wal.enable_log_recycle Bool true Whether to reuse logically truncated log files.
It's only used when the provider is raft_engine.
wal.prefill_log_files Bool false Whether to pre-create log files on start up.
It's only used when the provider is raft_engine.
wal.sync_period String 5s Duration for fsyncing log files.
It's only used when the provider is raft_engine.
wal.recovery_parallelism Integer Auto Parallelism during WAL recovery. Defaults to the number of CPU cores; this example sets it to 2.
wal.broker_endpoints Array -- The Kafka broker endpoints.
It's only used when the provider is kafka.
wal.connect_timeout String 3s The connect timeout for kafka client.
It's only used when the provider is kafka.
wal.timeout String 5s The total request timeout for kafka client.
It's only used when the provider is kafka.
wal.auto_create_topics Bool true Automatically create topics for WAL.
Set to true to automatically create topics for WAL.
Otherwise, use topics named topic_name_prefix_[0..num_topics)
wal.num_topics Integer 64 Number of topics.
It's only used when the provider is kafka.
wal.selector_type String round_robin Topic selector type.
Available selector types:
- round_robin (default)
It's only used when the provider is kafka.
wal.topic_name_prefix String greptimedb_wal_topic A Kafka topic is constructed by concatenating topic_name_prefix and topic_id.
i.g., greptimedb_wal_topic_0, greptimedb_wal_topic_1.
It's only used when the provider is kafka.
wal.replication_factor Integer 1 Expected number of replicas of each partition.
It's only used when the provider is kafka.
wal.create_topic_timeout String 30s Above which a topic creation operation will be cancelled.
It's only used when the provider is kafka.
wal.max_batch_bytes String 1MB The max size of a single producer batch.
Warning: Kafka has a default limit of 1MB per message in a topic.
Defaults to 8MB when the provider is experimental_object_store.
It's only used when the provider is kafka or experimental_object_store.
wal.consumer_wait_timeout String 100ms The consumer wait timeout.
It's only used when the provider is kafka.
wal.overwrite_entry_start_id Bool false Ignore missing entries during read WAL.
It's only used when the provider is kafka.

This option ensures that when Kafka messages are deleted, the system
can still successfully replay memtable data without throwing an
out-of-range error.
However, enabling this option might lead to unexpected data loss,
as the system will skip over missing entries instead of treating
them as critical errors.
wal.storage_provider String "" The name of the storage provider that holds the WAL objects, an empty name selects the default object store.
It's only used when the provider is experimental_object_store.
wal.prefix String wal The path prefix of the WAL objects inside the storage provider.
The objects are written under <prefix>/datanodes/<node_id>/epochs/<generation>, which is derived from this prefix.
It's only used when the provider is experimental_object_store.
wal.flush_interval String 100ms The interval of flushing buffered entries to the object store, at least 10ms, defaults to 100ms.
Each non-empty timer-triggered flush creates one object, and a batch that reaches max_batch_bytes is flushed immediately, so under sustained load the batch seals on size and the interval no longer matters.
When writes are sparse, a shorter interval lowers the acknowledgement latency of durable appends and raises the number of object requests: timer-triggered sealing creates at most one object per interval per node, and in enqueued mode the backlog thresholds can seal earlier. An enqueued append does not wait for its batch to seal once it is admitted, but the backlog thresholds can delay admission.
It's only used when the provider is experimental_object_store.
wal.ack_mode String durable When an append to the object store WAL returns.
- durable: an append returns after the object holding its entries is durable (the default).
- enqueued: an append returns once its entries are admitted and their ids are assigned, and the object is created in the background. A crash loses the unpersisted backlog.
It's only used when the provider is experimental_object_store.
wal.max_unpersisted_bytes String 64MB The size of the unpersisted backlog at which new appends stall until an upload completes, in enqueued mode.
Nothing is dropped and nothing is rejected; the threshold does not bound what a crash during an outage can lose.
It's only used when the provider is experimental_object_store and ack_mode is enqueued.
wal.max_unpersisted_age String 8s The age of the oldest unpersisted entry at which new appends stall until an upload completes, in enqueued mode.
It's only used when the provider is experimental_object_store and ack_mode is enqueued.
wal.on_corrupted_segment String skip What a read does with a segment that still does not decode after a second fetch, because its checksum does not match or its content disagrees with its footer entry.
- skip: the segment is skipped and recorded as a WAL hole of its region, a metric is incremented and a warning is logged; the other regions of the object are unaffected (the default).
- fail: the read fails, so the region does not open.
It's only used when the provider is experimental_object_store.
metadata_store -- -- Metadata storage options.
metadata_store.file_size String 64MB The size of the metadata store log file.
metadata_store.purge_threshold String 256MB The threshold of the metadata store size to trigger a purge.
metadata_store.purge_interval String 1m The interval of the metadata store to trigger a purge.
procedure -- -- Procedure storage options.
procedure.max_retry_times Integer 3 Procedure max retry time.
procedure.retry_delay String 500ms Initial retry delay of procedures, increases exponentially
procedure.max_running_procedures Integer 128 Max running procedures.
The maximum number of procedures that can be running at the same time.
If the number of running procedures exceeds this limit, the procedure will be rejected.
query -- -- The query engine options.
query.parallelism Integer 0 Parallelism of the query engine.
Default to 0, which means the number of CPU cores.
query.memory_pool_size String 0 Memory pool size for query execution operators (aggregation, sorting, join).
Supports absolute size (e.g., "2GB", "4GB") or percentage of system memory (e.g., "20%").
Setting it to 0 disables the limit. Defaults to 0 (unlimited); this example sets it to 50%.
When this limit is reached, queries will fail with ResourceExhausted error.
NOTE: This does NOT limit memory used by table scans.
query.experimental_memory_pool_policy String greedy Experimental memory pool allocation policy:
- "greedy" (default): first-come-first-served allocation; preserves current behavior.
- "fair": divides available memory among spillable operators and may spill earlier.
Only effective when memory_pool_size is bounded (>0).
query.experimental_spill_mode String default Spill mode:
- "default": preserve DataFusion built-in OS temp directory (default).
- "custom": explicitly configure spill path, quota, and compression.
- "disabled": explicitly disable disk spilling.
Set this to "custom" before using the path/quota/compression keys below.
query.experimental_spill_path String Unset Spill directory path. Ignored unless mode is "custom".
query.experimental_spill_max_temp_directory_size String 1GiB Maximum total size of spill directory (default: "1GiB").
Ignored unless mode is "custom".
query.experimental_spill_compression String uncompressed Compression for spilled data files: "uncompressed" (default), "lz4_frame", "zstd".
Ignored unless mode is "custom".
storage -- -- The data storage options.
storage.data_home String ./greptimedb_data The working home directory.
storage.copy_root String ./greptimedb_data/copy Root directory for standalone SQL access to local files.
Relative SQL paths are resolved below this directory. Absolute paths are accepted only when
they are inside this directory. Defaults to <data_home>/copy.
Distributed deployments always reject SQL access to local files.
Upgrade note: COPY commands and existing external tables that reference paths outside this
directory will fail. Move those files below the copy root, set this option to a dedicated
directory containing them, or migrate the files to object storage before upgrading.
storage.type String File The storage type used to store the data.
- File: the data is stored in the local file system.
- S3: the data is stored in the S3 object storage.
- Gcs: the data is stored in the Google Cloud Storage.
- Azblob: the data is stored in the Azure Blob Storage.
- Oss: the data is stored in the Aliyun OSS.
- Hdfs: the data is stored in the Hadoop Distributed File System.
storage.bucket String Unset The S3 bucket name.
It's only used when the storage type is S3, Oss and Gcs.
storage.root String Unset The directory or object prefix under which data is stored.
It's only used when the storage type is S3, Oss, Gcs, Azblob and Hdfs.
storage.name_node String Unset The HDFS NameNode URI, for example, hdfs://127.0.0.1:9000.
It's only used when the storage type is Hdfs.
storage.options InlineTable Unset Additional options passed to the native HDFS client.
It's only used when the storage type is Hdfs.
storage.access_key_id String Unset The access key id of the aws account.
It's highly recommended to use AWS IAM roles instead of hardcoding the access key id and secret key.
It's only used when the storage type is S3 and Oss.
storage.secret_access_key String Unset The secret access key of the aws account.
It's highly recommended to use AWS IAM roles instead of hardcoding the access key id and secret key.
It's only used when the storage type is S3.
storage.access_key_secret String Unset The secret access key of the aliyun account.
It's only used when the storage type is Oss.
storage.account_name String Unset The account key of the azure account.
It's only used when the storage type is Azblob.
storage.account_key String Unset The account key of the azure account.
It's only used when the storage type is Azblob.
storage.scope String Unset The scope of the google cloud storage.
It's only used when the storage type is Gcs.
storage.credential_path String Unset The credential path of the google cloud storage.
It's only used when the storage type is Gcs.
storage.credential String Unset The credential of the google cloud storage.
It's only used when the storage type is Gcs.
storage.container String Unset The container of the azure account.
It's only used when the storage type is Azblob.
storage.sas_token String Unset The sas token of the azure account.
It's only used when the storage type is Azblob.
storage.endpoint String Unset The endpoint of the S3 service.
It's only used when the storage type is S3, Oss, Gcs and Azblob.
storage.region String Unset The region of the S3 service.
It's only used when the storage type is S3, Oss, Gcs and Azblob.
storage.http_client -- -- The http client options to the storage.
It's only used when the storage type is S3, Oss, Gcs and Azblob.
storage.http_client.pool_max_idle_per_host Integer 1024 The maximum idle connection per host allowed in the pool.
storage.http_client.connect_timeout String 30s The timeout for only the connect phase of a http client.
storage.http_client.timeout String 30s The total request timeout, applied from when the request starts connecting until the response body has finished.
Also considered a total deadline.
storage.http_client.pool_idle_timeout String 90s The timeout for idle sockets being kept-alive.
storage.http_client.skip_ssl_validation Bool false To skip the ssl verification
Security Notice: Setting skip_ssl_validation = true disables certificate verification, making connections vulnerable to man-in-the-middle attacks. Only use this in development or trusted private networks.
[[region_engine]] -- -- The region engine options. You can configure multiple region engines.
Each engine type (mito, file, metric) may appear only once; duplicates cause startup to fail.
region_engine.mito -- -- The Mito engine options.
region_engine.mito.num_workers Integer Auto Number of region workers. Defaults to half of the available CPU cores, rounded up.
region_engine.mito.worker_channel_size Integer 128 Request channel size of each worker.
region_engine.mito.worker_request_batch_size Integer 64 Max batch size for a worker to handle requests.
region_engine.mito.manifest_checkpoint_distance Integer 10 Number of meta action updated to trigger a new checkpoint for the manifest.
region_engine.mito.compress_manifest Bool false Whether to compress manifest and checkpoint file by gzip (default false).
region_engine.mito.experimental_enable_series_index Bool false Under development; do not enable. Whether to enable series indexes.
Indexes are stored on the local filesystem under {data_home}/series_index.
region_engine.mito.experimental_series_index_max_size String 5GiB Approximate series and range index size limit in open regions, shared across workers.
Workers periodically refresh usage and skip maintenance when full. In-flight reconciliation
can exceed the limit. Closed-region files, temporary output, catalogs, and old snapshots
retained by readers are not counted.
Minimum: 1KiB. Takes effect on restart.
region_engine.mito.experimental_enable_range_index Bool false Whether to build and query range indexes when series indexes are enabled.
Obsolete range-index metadata and files are still cleaned up when disabled.
region_engine.mito.experimental_series_index_maintenance_interval String 5m Interval between series-index maintenance runs. Zero uses the default of 5 min.
region_engine.mito.experimental_series_index_bucket_width String 5days Requested minimum series-index bucket width (default: 5 days), rounded up to
an exact multiple of each region's compaction time window.
region_engine.mito.max_background_flushes Integer Auto Max number of running background flush jobs (default: 1/2 of cpu cores).
region_engine.mito.max_background_compactions Integer Auto Max number of running background compaction jobs (default: 1/4 of cpu cores).
region_engine.mito.max_background_purges Integer Auto Max number of running background purge jobs (default: number of cpu cores).
region_engine.mito.experimental_compaction_memory_limit String 0 Memory budget for compaction tasks.
Supports absolute size (e.g., "2GiB", "512MB") or percentage of system memory (e.g., "50%").
Setting it to 0 or "unlimited" disables the limit.
region_engine.mito.experimental_compaction_on_exhausted String wait Behavior when compaction cannot acquire memory from the budget.
Options: "wait" (default, 10s), "wait()", "fail"
region_engine.mito.auto_flush_interval String 10m Interval to auto flush a region if it has not flushed yet.
region_engine.mito.global_write_buffer_size String Auto Global write buffer size for all regions. If not set, it's default to 1/8 of OS memory with a max limitation of 1GB.
region_engine.mito.global_write_buffer_reject_size String Auto Global write buffer size threshold to reject write requests. If not set, it's default to 2 times of global_write_buffer_size.
region_engine.mito.default_region_write_buffer_size String 0 Default write buffer size for each region. Regions stall at this size and reject writes at twice this size. Setting it to 0 disables both limits unless the table specifies write_buffer_size.
region_engine.mito.sst_meta_cache_size String Auto Cache size for SST metadata. Setting it to 0 to disable the cache.
If not set, it's default to 1/8 of OS memory with a max limitation of 512MB.
region_engine.mito.vector_cache_size String Auto Cache size for vectors and arrow arrays. Setting it to 0 to disable the cache.
If not set, it's default to 1/16 of OS memory with a max limitation of 512MB.
region_engine.mito.page_cache_size String Auto Cache size for pages of SST row groups. Setting it to 0 to disable the cache.
If not set, it's default to 1/8 of OS memory.
region_engine.mito.selector_result_cache_size String Auto Cache size for time series selector (e.g. last_value()). Setting it to 0 to disable the cache.
If not set, it's default to 1/16 of OS memory with a max limitation of 512MB.
region_engine.mito.range_result_cache_size String Auto Cache size for flat range scan results. Setting it to 0 to disable the cache.
If not set, it's default to 1/16 of OS memory with a max limitation of 512MB.
region_engine.mito.prefilter_result_cache_size String Auto Cache size for prefilter results. Setting it to 0 to disable the cache.
If not set, it's default to 1/32 of OS memory with a max limitation of 128MB.
region_engine.mito.enable_write_cache Bool false Whether to enable the write cache, it's enabled by default when using object storage. It is recommended to enable it when using object storage for better performance.
region_engine.mito.write_cache_path String "" File system path for write cache, defaults to {data_home}.
region_engine.mito.write_cache_size String 5GiB Capacity for write cache. If your disk space is sufficient, it is recommended to set it larger.
region_engine.mito.write_cache_ttl String Unset TTL for write cache.
region_engine.mito.preload_index_cache Bool true Preload index (puffin) files into cache on region open (default: true).
When enabled, index files are loaded into the write cache during region initialization,
which can improve query performance at the cost of longer startup times.
region_engine.mito.index_cache_percent Integer 20 Percentage of write cache capacity allocated for index (puffin) files (default: 20).
The remaining capacity is used for data (parquet) files.
Must be between 0 and 100 (exclusive). For example, with a 5GiB write cache and 20% allocation,
1GiB is reserved for index files and 4GiB for data files.
region_engine.mito.enable_refill_cache_on_read Bool true Enable refilling cache on read operations (default: true).
When disabled, cache refilling on read won't happen.
region_engine.mito.manifest_cache_size String 256MB Capacity for manifest cache (default: 256MB).
region_engine.mito.sst_write_buffer_size String 8MB Buffer size for SST writing.
region_engine.mito.max_concurrent_scan_files Integer 384 Maximum number of SST files to scan concurrently.
region_engine.mito.allow_stale_entries Bool false Whether to allow stale WAL entries read during replay.
region_engine.mito.scan_memory_limit String unlimited Memory limit for table scans across all queries.
Supports absolute size (e.g., "2GB") or percentage of system memory (e.g., "20%").
Setting it to 0 or "unlimited" disables the limit.
region_engine.mito.scan_memory_on_exhausted String fail Controls what happens when a scan cannot get memory immediately.
"fail" (default) fails fast and is the recommended option for most users.
"wait" / "wait()" waits for memory to become available. This is mainly
for advanced tuning in bursty workloads where temporary contention is common and
higher latency is acceptable.
"wait" means "wait(10s)", not unlimited waiting.
region_engine.mito.min_compaction_interval String 0m Minimum time interval between two compactions.
To align with the old behavior, the default value is 0 (no restrictions).
region_engine.mito.schedule_compaction_after_edit Bool true Whether to allow to schedule a compaction after a successful region edit.

Setting this to "true" is a necessary but not sufficient condition for scheduling compaction after a region edit.
Other constraints, such as "min_compaction_interval", may still prevent compaction from being scheduled.
Setting this to "false", however, guarantees that compaction will not be scheduled after a region edit.
region_engine.mito.default_flat_format Bool true Whether to enable flat format as the default SST format.
region_engine.mito.experimental_series_scan_v2 Bool false Whether to enable the experimental two-phase mode for eligible metric series scans.
region_engine.mito.index -- -- The options for index in Mito engine.
region_engine.mito.index.aux_path String "" Auxiliary directory path for the index in filesystem, used to store intermediate files for
creating the index and staging files for searching the index, defaults to {data_home}/index_intermediate.
The default name for this directory is index_intermediate for backward compatibility.

This path contains two subdirectories:
- __intm: for storing intermediate files used during creating index.
- staging: for storing staging files used during searching index.
region_engine.mito.index.staging_size String 2GB The max capacity of the staging directory.
region_engine.mito.index.staging_ttl String 7d The TTL of the staging directory.
Defaults to 7 days.
Setting it to "0s" to disable TTL.
region_engine.mito.index.metadata_cache_size String 64MiB Cache size for inverted index metadata.
region_engine.mito.index.content_cache_size String 128MiB Cache size for inverted index content.
region_engine.mito.index.content_cache_page_size String 64KiB Page size for inverted index content cache.
region_engine.mito.index.result_cache_size String 128MiB Cache size for index result.
region_engine.mito.inverted_index -- -- The options for inverted index in Mito engine.
region_engine.mito.inverted_index.create_on_flush String auto Whether to create the index on flush.
- auto: automatically (default)
- disable: never
region_engine.mito.inverted_index.create_on_compaction String auto Whether to create the index on compaction.
- auto: automatically (default)
- disable: never
region_engine.mito.inverted_index.apply_on_query String auto Whether to apply the index on query
- auto: automatically (default)
- disable: never
region_engine.mito.inverted_index.mem_threshold_on_create String auto Memory threshold for performing an external sort during index creation.
- auto: automatically determine the threshold based on the system memory size (default)
- unlimited: no memory limit
- [size] e.g. 64MB: fixed memory threshold
region_engine.mito.inverted_index.intermediate_path String "" Deprecated, use region_engine.mito.index.aux_path instead.
region_engine.mito.fulltext_index -- -- The options for full-text index in Mito engine.
region_engine.mito.fulltext_index.create_on_flush String auto Whether to create the index on flush.
- auto: automatically (default)
- disable: never
region_engine.mito.fulltext_index.create_on_compaction String auto Whether to create the index on compaction.
- auto: automatically (default)
- disable: never
region_engine.mito.fulltext_index.apply_on_query String auto Whether to apply the index on query
- auto: automatically (default)
- disable: never
region_engine.mito.fulltext_index.mem_threshold_on_create String auto Memory threshold for index creation.
- auto: automatically determine the threshold based on the system memory size (default)
- unlimited: no memory limit
- [size] e.g. 64MB: fixed memory threshold
region_engine.mito.bloom_filter_index -- -- The options for bloom filter in Mito engine.
region_engine.mito.bloom_filter_index.create_on_flush String auto Whether to create the bloom filter on flush.
- auto: automatically (default)
- disable: never
region_engine.mito.bloom_filter_index.create_on_compaction String auto Whether to create the bloom filter on compaction.
- auto: automatically (default)
- disable: never
region_engine.mito.bloom_filter_index.apply_on_query String auto Whether to apply the bloom filter on query
- auto: automatically (default)
- disable: never
region_engine.mito.bloom_filter_index.mem_threshold_on_create String auto Memory threshold for bloom filter creation.
- auto: automatically determine the threshold based on the system memory size (default)
- unlimited: no memory limit
- [size] e.g. 64MB: fixed memory threshold
region_engine.file -- -- Enable the file engine.
region_engine.metric -- -- Metric engine options.
logging -- -- The logging options.
logging.dir String ./greptimedb_data/logs The directory to store the log files. If set to empty, logs will not be written to files.
logging.level String Unset The log level. Can be info/debug/warn/error.
logging.enable_otlp_tracing Bool false Enable OTLP tracing.
logging.otlp_endpoint String http://localhost:4318/v1/traces The OTLP tracing endpoint.
logging.append_stdout Bool true Whether to append logs to stdout.
logging.enable_file_logging Bool true Whether to write logs to files in dir.
logging.log_format String text The log format. Can be text/json.
logging.max_log_files Integer 720 The maximum amount of log files.
logging.max_log_dir_size String 0B The maximum total size of managed log files in dir.
Old closed log files are removed before writing when necessary. Active files may exceed it. Set to 0B to disable.
logging.enable_per_region_metrics Bool false Whether to enable per-region metrics.
Default to false.
logging.otlp_export_protocol String http The OTLP tracing export protocol. Can be grpc/http.
logging.otlp_headers -- -- Additional OTLP headers, only valid when using OTLP http
logging.tracing_sample_ratio -- Unset The percentage of tracing will be sampled and exported.
Valid range [0, 1], 1 means all traces are sampled, 0 means all traces are not sampled, the default value is 1.
ratio > 1 are treated as 1. Fractions < 0 are treated as 0
logging.tracing_sample_ratio.default_ratio Float 1.0 --
slow_query -- -- The slow query log options.
slow_query.enable Bool true Whether to enable slow query log.
slow_query.record_type String system_table The record type of slow queries. It can be system_table or log.
If system_table is selected, the slow queries will be recorded in a system table greptime_private.slow_queries.
If log is selected, the slow queries will be logged in a log file greptimedb-slow-queries.*.
slow_query.threshold String 30s The threshold of slow query. It can be human readable time string, for example: 10s, 100ms, 1s.
slow_query.sample_ratio Float 1.0 The sampling ratio of slow query log. The value should be in the range of (0, 1]. For example, 0.1 means 10% of the slow queries will be logged and 1.0 means all slow queries will be logged.
slow_query.ttl String 90d The TTL of the slow_queries system table. Default is 90d when record_type is system_table.
tracing -- -- The tracing options. Only effect when compiled with tokio-console feature.
tracing.tokio_console_addr String Unset The tokio console address.
pipeline -- -- The pipeline options.
pipeline.cache_ttl String 10s Time to live of the local pipeline cache. Default is 10s.
event_recorder -- -- Configuration options for the event recorder.
event_recorder.ttl String 90d TTL for the events table that will be used to store the events. Default is 90d.
event_recorder.event_types Array All Event types to record. Current available event types: create_database,
alter_database, drop_database, create_flow, drop_flow,
create_table, create_logical_tables, alter_table, alter_logical_tables,
drop_table, undrop_table, purge_dropped_table, truncate_table,
create_view, drop_view, admin_function, reconcile_catalog,
reconcile_database, reconcile_table, reconcile_logical_tables.
When omitted, all current and future event types are recorded.
Set to an empty array to disable event recording.
memory -- -- The memory options.
memory.enable_heap_profiling Bool true Whether to enable heap profiling activation during startup.
When enabled, heap profiling will be activated if the MALLOC_CONF environment variable
is set to "prof:true,prof_active:false". The official image adds this env variable.
Default is true.

Distributed Mode

Frontend

Key Type Default Descriptions
default_timezone String Unset The default timezone of the server.
default_column_prefix String Unset The default column prefix for auto-created time index, value, and native histogram columns.
Legacy OTLP summary columns keep their historical greptime_ prefix.
auto_create_table Bool true Server-side global switch for auto table creation on write.
When false, a missing table is never auto-created even if the request sets the auto_create_table hint to true. Default: true.
experimental_metric_export Bool false Enables experimental Parquet COPY DATABASE using shared Metric physical scans.
Resume requires the previous export and storage writes to have ended; HTTP timeout is not confirmation.
user_provider String Unset The user provider for authentication.
Examples: "static_user_provider:file:/path/to/users", "static_user_provider:cmd:greptime_user=greptime_pwd"
Password verifier formats: "plain:", "pbkdf2_sha256::<hex_salt>:<hex_hash>",
"mysql_native_password:<hex_sha1_sha1_password>",
"pg_scram_sha256::<hex_salt>:<hex_stored_key>:<hex_server_key>"
"pbkdf2_sha256" and "pg_scram_sha256" protect passwords at rest, but such users cannot log in over MySQL:
static_user_provider and watch_file_user_provider only negotiate mysql_native_password, which cannot verify these formats.
"mysql_native_password" is MySQL-specific and cannot authenticate over PostgreSQL at all.
PostgreSQL SCRAM only covers "plain" and "pg_scram_sha256" users; if any user is "pbkdf2_sha256" or
"mysql_native_password", PostgreSQL falls back to cleartext password auth for every user.
For "pg_scram_sha256" users, keep the default iteration count (4096) and salt length (16): both are
observable in the SCRAM server-first message, and non-default values weaken resistance to username
enumeration.
max_in_flight_write_bytes String 0 Maximum total memory for all concurrent write request bodies and messages (HTTP, gRPC, Flight).
Set to 0 to disable the limit. Default: "0" (unlimited)
write_bytes_exhausted_policy String wait Policy when write bytes quota is exhausted.
Options: "wait" (default, 10s timeout), "wait()" (e.g., "wait(30s)"), "fail"
runtime -- -- The runtime options.
runtime.global_rt_size Integer Auto The number of threads to execute the runtime for global read operations.
Defaults to max(num_cpus, 2).
runtime.compact_rt_size Integer Auto The number of threads to execute compact operations.
Defaults to max(num_cpus / 2, 2).
runtime.compact_rt_max_blocking_threads Integer Auto The maximum number of blocking threads for compact operations.
Defaults to max(num_cpus / 2, 2).
http -- -- The HTTP server options.
http.addr String 127.0.0.1:4000 The address to bind the HTTP server.
http.timeout String 0s HTTP request timeout. Set to 0 to disable timeout.
When synchronous Prometheus, OTLP metrics, or ordinary-table batching is enabled, a nonzero timeout is
raised to at least the largest active flush interval plus 1 second. The intervals come from
pending_rows_batcher.logical_table.pending_rows_flush_interval and pending_rows_batcher.pending_rows_flush_interval.
http.body_limit String 64MB HTTP request body limit.
The following units are supported: B, KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB.
Set to 0 to disable limit.
http.enable_cors Bool true HTTP CORS support, it's turned on by default
This allows browser to access http APIs without CORS restrictions
http.cors_allowed_origins Array Unset Customize allowed origins for HTTP CORS.
http.enable_api_server Bool false Whether to start the dedicated public HTTP API server. This server serves
only the v1 interfaces plus the dashboard, and shares every other [http]
option with the main server. It is disabled by default; set to true to enable.
http.api_server_addr String 127.0.0.1:4006 The address to bind the dedicated HTTP API server, in the same form as addr.
Defaults to 127.0.0.1:4006.
grpc -- -- The gRPC server options.
grpc.bind_addr String 127.0.0.1:4001 The address to bind the gRPC server.
grpc.server_addr String Unset The address advertised to the metasrv, and used for connections from outside the host.
If left empty or unset, the server will automatically use the IP address of the first network interface
on the host, with the same port number as the one specified in grpc.bind_addr.
grpc.runtime_size Integer 8 The number of server worker threads.
grpc.flight_compression String arrow_ipc Compression mode for frontend side Arrow IPC service. Available options:
- none: disable all compression
- transport: only enable gRPC transport compression (zstd)
- arrow_ipc: only enable Arrow IPC compression (lz4)
- all: enable all compression.
Default to arrow_ipc.
grpc.max_connection_age String Unset The maximum connection age for gRPC connection.
The value can be a human-readable time string. For example: 10m for ten minutes or 1h for one hour.
Refer to https://grpc.io/docs/guides/keepalive/ for more details.
grpc.tls -- -- gRPC server TLS options, see mysql.tls section.
grpc.tls.mode String disable TLS mode.
grpc.tls.cert_path String Unset Certificate file path.
grpc.tls.key_path String Unset Private key file path.
grpc.tls.watch Bool false Watch for Certificate and key file change and auto reload.
For now, gRPC tls config does not support auto reload.
internal_grpc -- -- The internal gRPC server options. Internal gRPC port for nodes inside cluster to access frontend.
The internal gRPC server starts only when this section is set.
internal_grpc.bind_addr String Unset The address to bind the gRPC server.
If unset, it falls back to 127.0.0.1:4001, the same address as grpc.bind_addr.
internal_grpc.server_addr String Unset The address advertised to the metasrv, and used for connections from outside the host.
Unlike grpc.server_addr, it is not detected automatically; set it explicitly.
internal_grpc.runtime_size Integer 8 The number of server worker threads.
internal_grpc.flight_compression String arrow_ipc Compression mode for frontend side Arrow IPC service. Available options:
- none: disable all compression
- transport: only enable gRPC transport compression (zstd)
- arrow_ipc: only enable Arrow IPC compression (lz4)
- all: enable all compression.
Default to arrow_ipc.
internal_grpc.tls -- -- internal gRPC server TLS options, see mysql.tls section.
internal_grpc.tls.mode String disable TLS mode.
internal_grpc.tls.cert_path String Unset Certificate file path.
internal_grpc.tls.key_path String Unset Private key file path.
internal_grpc.tls.watch Bool false Watch for Certificate and key file change and auto reload.
For now, gRPC tls config does not support auto reload.
mysql -- -- MySQL server options.
mysql.enable Bool true Whether to enable.
mysql.addr String 127.0.0.1:4002 The addr to bind the MySQL server.
mysql.runtime_size Integer 2 The number of server worker threads.
mysql.keep_alive String 0s Server-side keep-alive time.
Set to 0 (default) to disable.
mysql.prepared_stmt_cache_size Integer 10000 Maximum entries in the MySQL prepared statement cache; default is 10,000.
mysql.tls -- -- --
mysql.tls.mode String disable TLS mode, refer to https://www.postgresql.org/docs/current/libpq-ssl.html
- disable (default value)
- prefer
- require
- verify-ca
- verify-full
mysql.tls.cert_path String Unset Certificate file path.
mysql.tls.key_path String Unset Private key file path.
mysql.tls.watch Bool false Watch for Certificate and key file change and auto reload
postgres -- -- PostgresSQL server options.
postgres.enable Bool true Whether to enable
postgres.addr String 127.0.0.1:4003 The addr to bind the PostgresSQL server.
postgres.runtime_size Integer 2 The number of server worker threads.
postgres.keep_alive String 0s Server-side keep-alive time.
Set to 0 (default) to disable.
postgres.tls -- -- PostgresSQL server TLS options, see mysql.tls section.
postgres.tls.mode String disable TLS mode.
postgres.tls.cert_path String Unset Certificate file path.
postgres.tls.key_path String Unset Private key file path.
postgres.tls.watch Bool false Watch for Certificate and key file change and auto reload
opentsdb -- -- OpenTSDB protocol options.
opentsdb.enable Bool true Whether to enable OpenTSDB put in HTTP API.
influxdb -- -- InfluxDB protocol options.
influxdb.enable Bool true Whether to enable InfluxDB protocol in HTTP API.
influxdb.default_merge_mode String last_non_null Default merge mode for tables automatically created by InfluxDB protocol.
Available values: "last_non_null", "last_row".
pending_rows_batcher -- -- Ordinary-table batching for opted-in ingestion protocols.
PENDING_ROWS_BATCH_SYNC defaults to true for both batchers. Set it to false to acknowledge
queue admission without waiting for storage; later failures cannot be returned to the client.
Omitted or empty protocols disables batching. Prom without metric engine uses this batcher.
OTLP logs, traces and ordinary metrics use this batcher.
MySQL and PostgreSQL use the same acknowledgement policy. Protocol timeouts are unchanged.
Single-connection writes and INSERT SELECT may incur additional flush waits.
pending_rows_batcher.pending_rows_flush_interval String 0s Flush interval measured from the first pending submission. Zero disables batching.
pending_rows_batcher.max_batch_rows Integer 100000 Flush after a complete submission reaches this row threshold.
pending_rows_batcher.max_concurrent_flushes Integer 256 Maximum concurrent flushes shared by the frontend batcher.
pending_rows_batcher.worker_channel_capacity Integer 65536 Maximum queued submissions per table worker.
pending_rows_batcher.max_inflight_requests Integer 3000 Maximum admitted original requests awaiting completion.
pending_rows_batcher.flow_notification_queue_capacity Integer 1024 Maximum number of queued table Flow notifications.
pending_rows_batcher.logical_table -- -- Metric-engine logical-table batching for Prom remote write and non-legacy OTLP metrics.
Requires prom_store.with_metric_engine. Logs, traces and legacy metrics are not eligible.
Enable independently with protocols and a nonzero flush interval.
Omitted fields use independent defaults, not parent settings.
Empty protocols or a zero interval disables logical batching without fallback.
Omitting this entire section preserves legacy Prom batching; it does not enable OTLP batching.
pending_rows_batcher.logical_table.pending_rows_flush_interval String 0s Flush interval measured from the first pending submission. Zero disables batching.
pending_rows_batcher.logical_table.max_batch_rows Integer 100000 Flush after a complete submission reaches this row threshold.
pending_rows_batcher.logical_table.max_concurrent_flushes Integer 256 Maximum concurrent flushes shared by Prom and OTLP metrics.
pending_rows_batcher.logical_table.worker_channel_capacity Integer 65536 Maximum queued submissions per physical-table worker.
pending_rows_batcher.logical_table.max_inflight_requests Integer 3000 Maximum admitted original requests awaiting completion.
pending_rows_batcher.logical_table.flow_notification_queue_capacity Integer 1024 Maximum number of queued logical-table Flow notifications.
jaeger -- -- Jaeger protocol options.
jaeger.enable Bool true Whether to enable Jaeger protocol in HTTP API.
otlp -- -- OpenTelemetry protocol options.
otlp.enable Bool true Whether to enable OpenTelemetry protocol in HTTP API.
otlp.trace_ingest_chunk_size Integer 512 Maximum spans per trace ingest chunk. Set to 0 to disable splitting.
otlp.experimental_enable_resource_info Bool false Whether to synthesize the greptime_otel_resource_info table from OTLP metric
resource attributes, so metrics-only services reach the semantic graph.
prom_store -- -- Prometheus remote storage options
prom_store.enable Bool true Whether to enable Prometheus remote write and read in HTTP API.
prom_store.with_metric_engine Bool true Whether to store the data from Prometheus remote write in metric engine.
prom_store.prom_validation_mode String strict Whether to enable validation for Prometheus remote write requests.
Available options:
- strict: deny invalid UTF-8 strings (default).
- lossy: allow invalid UTF-8 strings, replace invalid characters with REPLACEMENT_CHARACTER(U+FFFD).
- unchecked: do not valid strings.
meta_client -- -- The metasrv client options.
meta_client.metasrv_addrs Array -- The addresses of the metasrv.
meta_client.timeout String 3s Operation timeout.
meta_client.ddl_timeout String 10s DDL timeout.
meta_client.connect_timeout String 1s Connect server timeout.
meta_client.tcp_nodelay Bool true TCP_NODELAY option for accepted connections.
meta_client.metadata_cache_max_capacity Integer 100000 The configuration about the cache of the metadata.
meta_client.metadata_cache_ttl String 10m TTL of the metadata cache.
meta_client.metadata_cache_tti String 5m --
query -- -- The query engine options.
query.parallelism Integer 0 Parallelism of the query engine.
Default to 0, which means the number of CPU cores.
query.allow_query_fallback Bool false Whether to allow query fallback when push down optimize fails.
Default to false, meaning when push down optimize failed, return error msg
query.memory_pool_size String 0 Memory pool size for query execution operators (aggregation, sorting, join).
Supports absolute size (e.g., "4GB", "8GB") or percentage of system memory (e.g., "30%").
Setting it to 0 disables the limit. Defaults to 0 (unlimited); this example sets it to 50%.
When this limit is reached, queries will fail with ResourceExhausted error.
NOTE: This does NOT limit memory used by table scans (only applies to datanodes).
query.experimental_memory_pool_policy String greedy Experimental memory pool allocation policy:
- "greedy" (default): first-come-first-served allocation; preserves current behavior.
- "fair": divides available memory among spillable operators and may spill earlier.
Only effective when memory_pool_size is bounded (>0).
query.experimental_spill_mode String default Spill mode:
- "default": preserve DataFusion built-in OS temp directory (default).
- "custom": explicitly configure spill path, quota, and compression.
- "disabled": explicitly disable disk spilling.
Set this to "custom" before using the path/quota/compression keys below.
query.experimental_spill_path String Unset Spill directory path. Ignored unless mode is "custom".
query.experimental_spill_max_temp_directory_size String 1GiB Maximum total size of spill directory (default: "1GiB").
Ignored unless mode is "custom".
query.experimental_spill_compression String uncompressed Compression for spilled data files: "uncompressed" (default), "lz4_frame", "zstd".
Ignored unless mode is "custom".
write_retry -- -- Retry retryable distributed write failures.
Backoff grows exponentially, with jitter to spread concurrent retries.
write_retry.max_retries Integer 3 Maximum retries after the initial write attempt. Set to 0 to disable retries.
write_retry.initial_backoff String 1500ms Initial backoff before jitter is applied. Must be positive.
write_retry.max_backoff String 10s Maximum backoff before jitter is applied. Must be at least initial_backoff.
write_retry.backoff_multiplier Float 2.0 Backoff multiplier. Must be finite and at least 1.
datanode -- -- Datanode options.
datanode.client -- -- Datanode client options.
datanode.client.connect_timeout String 1s Connect server timeout. Defaults to 1s; this example sets it to 10s.
datanode.client.tcp_nodelay Bool true --
datanode.client.max_recv_message_size String 512MB The maximum receive message size for the gRPC client.
datanode.client.max_send_message_size String 512MB The maximum send message size for the gRPC client.
logging -- -- The logging options.
logging.dir String ./greptimedb_data/logs The directory to store the log files. If set to empty, logs will not be written to files.
logging.level String Unset The log level. Can be info/debug/warn/error.
logging.enable_otlp_tracing Bool false Enable OTLP tracing.
logging.otlp_endpoint String http://localhost:4318/v1/traces The OTLP tracing endpoint.
logging.append_stdout Bool true Whether to append logs to stdout.
logging.enable_file_logging Bool true Whether to write logs to files in dir.
logging.log_format String text The log format. Can be text/json.
logging.max_log_files Integer 720 The maximum amount of log files.
logging.max_log_dir_size String 0B The maximum total size of managed log files in dir.
Old closed log files are removed before writing when necessary. Active files may exceed it. Set to 0B to disable.
logging.enable_per_region_metrics Bool false Whether to enable per-region metrics.
Default to false.
logging.otlp_export_protocol String http The OTLP tracing export protocol. Can be grpc/http.
logging.otlp_headers -- -- Additional OTLP headers, only valid when using OTLP http
logging.tracing_sample_ratio -- Unset The percentage of tracing will be sampled and exported.
Valid range [0, 1], 1 means all traces are sampled, 0 means all traces are not sampled, the default value is 1.
ratio > 1 are treated as 1. Fractions < 0 are treated as 0
logging.tracing_sample_ratio.default_ratio Float 1.0 --
slow_query -- -- The slow query log options.
slow_query.enable Bool true Whether to enable slow query log.
slow_query.record_type String system_table The record type of slow queries. It can be system_table or log.
If system_table is selected, the slow queries will be recorded in a system table greptime_private.slow_queries.
If log is selected, the slow queries will be logged in a log file greptimedb-slow-queries.*.
slow_query.threshold String 30s The threshold of slow query. It can be human readable time string, for example: 10s, 100ms, 1s.
slow_query.sample_ratio Float 1.0 The sampling ratio of slow query log. The value should be in the range of (0, 1]. For example, 0.1 means 10% of the slow queries will be logged and 1.0 means all slow queries will be logged.
slow_query.ttl String 90d The TTL of the slow_queries system table. Default is 90d when record_type is system_table.
tracing -- -- The tracing options. Only effect when compiled with tokio-console feature.
tracing.tokio_console_addr String Unset The tokio console address.
memory -- -- The memory options.
memory.enable_heap_profiling Bool true Whether to enable heap profiling activation during startup.
When enabled, heap profiling will be activated if the MALLOC_CONF environment variable
is set to "prof:true,prof_active:false". The official image adds this env variable.
Default is true.
pipeline -- -- The pipeline options.
pipeline.cache_ttl String 10s Time to live of the frontend-local pipeline cache. A pipeline created or deleted on
another frontend takes effect on this one after at most this duration.
event_recorder -- -- Configuration options for the event recorder.
event_recorder.ttl String 90d TTL for the events table that will be used to store the events. Default is 90d.
event_recorder.event_types Array All Event types to record. Current available event type: admin_function.
When omitted, all current and future event types are recorded.
Set to an empty array to disable event recording.

Metasrv

Key Type Default Descriptions
data_home String ./greptimedb_data The working home directory.
store_addrs Array -- Store server address(es). The format depends on the selected backend.

For etcd: a list of "host:port" endpoints.
e.g. ["192.168.1.1:2379", "192.168.1.2:2379"]

For PostgreSQL: a connection string in libpq format or URI.
e.g.
- "host=localhost port=5432 user=postgres password= dbname=postgres"
- "postgresql://user:password@localhost:5432/mydb?connect_timeout=10"
The detail see: https://docs.rs/tokio-postgres/latest/tokio_postgres/config/struct.Config.html

For mysql store, the format is a MySQL connection URL.
e.g. "mysql://user:password@localhost:3306/greptime_meta?ssl-mode=VERIFY_CA&ssl-ca=/path/to/ca.pem"
store_key_prefix String "" If it's not empty, the metasrv will store all data with this key prefix.
backend String etcd_store The datastore for meta server.
Available values:
- etcd_store (default value)
- memory_store
- postgres_store
- mysql_store
meta_table_name String greptime_metakv Table name in RDS to store metadata. Effect when using a RDS kvbackend.
Only used when backend is postgres_store.
meta_schema_name String Unset Optional PostgreSQL schema for metadata table and election table name qualification.
When PostgreSQL public schema is not writable (e.g., PostgreSQL 15+ with restricted public),
set this to a writable schema. GreptimeDB will use meta_schema_name.meta_table_name.
Only used when backend is postgres_store.
Unset by default; this example sets it to greptime_schema.
auto_create_schema Bool true Automatically create PostgreSQL schema if it doesn't exist.
When enabled, the system will execute CREATE SCHEMA IF NOT EXISTS <schema_name>
before creating metadata tables. This is useful in production environments where
manual schema creation may be restricted.
Default is true.
Note: The PostgreSQL user must have CREATE SCHEMA permission for this to work.
Only used when backend is postgres_store.
meta_election_lock_id Integer 1 Advisory lock id in PostgreSQL for election. Effect when using PostgreSQL as kvbackend
Only used when backend is postgres_store.
selector String round_robin Datanode selector type.
- round_robin (default value)
- lease_based
- load_based
For details, please see "https://docs.greptime.com/developer-guide/metasrv/selector".
enable_region_failover Bool false Whether to enable region failover.
This feature is only available on GreptimeDB running on cluster mode and
- Using Remote WAL
- Using shared storage (e.g., s3).
region_failure_detector_initialization_delay String 10m The delay before starting region failure detection.
This delay helps prevent Metasrv from triggering unnecessary region failovers before all Datanodes are fully started.
Especially useful when the cluster is not deployed with GreptimeDB Operator and maintenance mode is not enabled.
allow_region_failover_on_local_wal Bool false Whether to allow region failover on local WAL.
This option is not recommended to be set to true, because it may lead to data loss during failover.
node_max_idle_time String 24hours Max allowed idle time before removing node info from metasrv memory.
heartbeat_interval String 3s Base heartbeat interval for calculating distributed time constants.
The frontend heartbeat interval is 6 times of the base heartbeat interval.
The flownode/datanode heartbeat interval is 1 times of the base heartbeat interval.
e.g., If the base heartbeat interval is 3s, the frontend heartbeat interval is 18s, the flownode/datanode heartbeat interval is 3s.
Heartbeat intervals are negotiated from metasrv during handshake; local node configs do not override this.
enable_telemetry Bool true Whether to enable greptimedb telemetry. Enabled by default.
runtime -- -- The runtime options.
runtime.global_rt_size Integer Auto The number of threads to execute the runtime for global read operations.
Defaults to max(num_cpus, 2).
runtime.compact_rt_size Integer Auto The number of threads to execute compact operations.
Defaults to max(num_cpus / 2, 2).
runtime.compact_rt_max_blocking_threads Integer Auto The maximum number of blocking threads for compact operations.
Defaults to max(num_cpus / 2, 2).
backend_tls -- -- TLS configuration for kv store backend (applicable for etcd, PostgreSQL, and MySQL backends)
When using etcd, PostgreSQL, or MySQL as metadata store, you can configure TLS here

Note: if TLS is configured in both this section and the store_addrs connection string, the
settings here will override the TLS settings in store_addrs.
backend_tls.mode String prefer TLS mode, refer to https://www.postgresql.org/docs/current/libpq-ssl.html
- "disable" - No TLS
- "prefer" (default) - Try TLS, fallback to plain
- "require" - Require TLS
- "verify_ca" - Require TLS and verify CA
- "verify_full" - Require TLS and verify hostname
backend_tls.cert_path String "" Path to client certificate file (for client authentication)
Like "/path/to/client.crt"
backend_tls.key_path String "" Path to client private key file (for client authentication)
Like "/path/to/client.key"
backend_tls.ca_cert_path String "" Path to CA certificate file (for server certificate verification)
Required when using custom CAs or self-signed certificates
Leave empty to use system root certificates only
Like "/path/to/ca.crt"
backend_client -- -- The backend client options.
Currently, only applicable when using etcd as the metadata store.
backend_client.keep_alive_timeout String 3s The keep alive timeout for backend client.
backend_client.keep_alive_interval String 10s The keep alive interval for backend client.
backend_client.connect_timeout String 3s The connect timeout for backend client.
grpc -- -- The gRPC server options.
grpc.bind_addr String 127.0.0.1:3002 The address to bind the gRPC server.
grpc.server_addr String Unset The communication server address for the frontend and datanode to connect to metasrv.
If left empty or unset, the server will automatically use the IP address of the first network interface
on the host, with the same port number as the one specified in grpc.bind_addr.
grpc.runtime_size Integer 8 The number of server worker threads.
grpc.max_recv_message_size String 512MB The maximum receive message size for gRPC server.
grpc.max_send_message_size String 512MB The maximum send message size for gRPC server.
grpc.http2_keep_alive_interval String 10s The server side HTTP/2 keep-alive interval
grpc.http2_keep_alive_timeout String 3s The server side HTTP/2 keep-alive timeout.
http -- -- The HTTP server options.
http.addr String 127.0.0.1:4000 The address to bind the HTTP server.
http.timeout String 0s HTTP request timeout. Set to 0 to disable timeout.
http.body_limit String 64MB HTTP request body limit.
The following units are supported: B, KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB.
Set to 0 to disable limit.
procedure -- -- Procedure storage options.
procedure.max_retry_times Integer 12 Procedure max retry time.
procedure.retry_delay String 500ms Initial retry delay of procedures, increases exponentially
procedure.max_metadata_value_size String 1500KiB Auto split large value
GreptimeDB procedure uses etcd as the default metadata storage backend.
The etcd the maximum size of any request is 1.5 MiB
1500KiB = 1536KiB (1.5MiB) - 36KiB (reserved size of key)
Comments out the max_metadata_value_size, for don't split large value (no limit).
procedure.max_running_procedures Integer 128 Max running procedures.
The maximum number of procedures that can be running at the same time.
If the number of running procedures exceeds this limit, the procedure will be rejected.
failure_detector -- -- --
failure_detector.threshold Float 8.0 Maximum acceptable φ before the peer is treated as failed.
Lower values react faster but yield more false positives.
failure_detector.min_std_deviation String 100ms The minimum standard deviation of the heartbeat intervals.
So tiny variations don’t make φ explode. Prevents hypersensitivity when heartbeat intervals barely vary.
failure_detector.acceptable_heartbeat_pause String 10000ms The acceptable pause duration between heartbeats.
Additional extra grace period to the learned mean interval before φ rises, absorbing temporary network hiccups or GC pauses.
datanode -- -- Datanode options.
datanode.client -- -- Datanode client options.
datanode.client.timeout String 10s Operation timeout.
datanode.client.connect_timeout String 1s Connect server timeout. Defaults to 1s; this example sets it to 10s.
datanode.client.tcp_nodelay Bool true TCP_NODELAY option for accepted connections.
datanode.client.max_recv_message_size String 512MB The maximum receive message size for the gRPC client.
datanode.client.max_send_message_size String 512MB The maximum send message size for the gRPC client.
wal -- -- --
wal.provider String raft_engine --
wal.broker_endpoints Array -- The broker endpoints of the Kafka cluster.

It's only used when the provider is kafka.
wal.connect_timeout String 3s The connect timeout for kafka client.
It's only used when the provider is kafka.
wal.timeout String 5s The total request timeout for kafka client.
It's only used when the provider is kafka.
wal.auto_create_topics Bool true Automatically create topics for WAL.
Set to true to automatically create topics for WAL.
Otherwise, use topics named topic_name_prefix_[0..num_topics)
It's only used when the provider is kafka.
wal.auto_prune_interval String 30m Interval of automatically WAL pruning.
Set to 0s to disable automatically WAL pruning which delete unused remote WAL entries periodically.
It's only used when the provider is kafka.
wal.auto_prune_logical_delete Bool false Whether automatically WAL pruning only updates the metadata marker and skips Kafka DeleteRecords.
Set to true for Kafka deployments that do not support DeleteRecords.
It's only used when the provider is kafka.
wal.flush_trigger_size String 512MB Estimated size threshold to trigger a flush when using Kafka remote WAL.
Since multiple regions may share a Kafka topic, the estimated size is calculated as:
(latest_entry_id - flushed_entry_id) * avg_record_size
MetaSrv triggers a flush for a region when this estimated size exceeds flush_trigger_size.
- latest_entry_id: The latest entry ID in the topic.
- flushed_entry_id: The last flushed entry ID for the region.
Set to "0" to let the system decide the flush trigger size.
It's only used when the provider is kafka.
wal.checkpoint_trigger_size String 128MB Estimated size threshold to trigger a checkpoint when using Kafka remote WAL.
The estimated size is calculated as:
(latest_entry_id - last_checkpoint_entry_id) * avg_record_size
MetaSrv triggers a checkpoint for a region when this estimated size exceeds checkpoint_trigger_size.
Set to "0" to let the system decide the checkpoint trigger size.
It's only used when the provider is kafka.
wal.auto_prune_parallelism Integer 10 Concurrent task limit for automatically WAL pruning.
It's only used when the provider is kafka.
wal.num_topics Integer 64 Number of topics used for remote WAL.
It's only used when the provider is kafka.
wal.selector_type String round_robin Topic selector type.
Available selector types:
- round_robin (default)
It's only used when the provider is kafka.
wal.topic_name_prefix String greptimedb_wal_topic A Kafka topic is constructed by concatenating topic_name_prefix and topic_id.
Only accepts strings that match the following regular expression pattern:
[a-zA-Z_:-][a-zA-Z0-9_:-.@#]*
i.g., greptimedb_wal_topic_0, greptimedb_wal_topic_1.
It's only used when the provider is kafka.
wal.replication_factor Integer 1 Expected number of replicas of each partition.
It's only used when the provider is kafka.
wal.create_topic_timeout String 30s The timeout for creating a Kafka topic.
It's only used when the provider is kafka.
event_recorder -- -- Configuration options for the event recorder.
event_recorder.ttl String 90d TTL for the events table that will be used to store the events. Default is 90d.
event_recorder.event_types Array All Event types to record. Current available event types: region_migration,
create_database, alter_database, drop_database, create_flow,
drop_flow, create_table, create_logical_tables, alter_table,
alter_logical_tables, drop_table, undrop_table, purge_dropped_table,
truncate_table, create_view, drop_view, repartition,
repartition_group, wal_prune, batch_gc, reconcile_catalog,
reconcile_database, reconcile_table, reconcile_logical_tables.
When omitted, all current and future event types are recorded.
Set to an empty array to disable event recording.
stats_persistence -- -- Configuration options for the stats persistence.
stats_persistence.ttl String 0s TTL for the stats table that will be used to store the stats.
Set to 0s to disable stats persistence.
Default is 0s.
If you want to enable stats persistence, set the TTL to a value greater than 0.
It is recommended to set a small value, e.g., 3h.
stats_persistence.interval String 10m The interval to persist the stats. Default is 10m.
The minimum value is 10m, if the value is less than 10m, it will be overridden to 10m.
gc -- -- --
gc.enable Bool false Whether GC is enabled. Default to false. Need to be the same with datanode's mito.gc.enable
If set to false, no GC will be performed
gc.gc_cooldown_period String 5m Cooldown period between GC operations on the same region.
logging -- -- The logging options.
logging.dir String ./greptimedb_data/logs The directory to store the log files. If set to empty, logs will not be written to files.
logging.level String Unset The log level. Can be info/debug/warn/error.
logging.enable_otlp_tracing Bool false Enable OTLP tracing.
logging.otlp_endpoint String http://localhost:4318/v1/traces The OTLP tracing endpoint.
logging.append_stdout Bool true Whether to append logs to stdout.
logging.enable_file_logging Bool true Whether to write logs to files in dir.
logging.log_format String text The log format. Can be text/json.
logging.max_log_files Integer 720 The maximum amount of log files.
logging.max_log_dir_size String 0B The maximum total size of managed log files in dir.
Old closed log files are removed before writing when necessary. Active files may exceed it. Set to 0B to disable.
logging.otlp_export_protocol String http The OTLP tracing export protocol. Can be grpc/http.
logging.otlp_headers -- -- Additional OTLP headers, only valid when using OTLP http
logging.tracing_sample_ratio -- Unset The percentage of tracing will be sampled and exported.
Valid range [0, 1], 1 means all traces are sampled, 0 means all traces are not sampled, the default value is 1.
ratio > 1 are treated as 1. Fractions < 0 are treated as 0
logging.tracing_sample_ratio.default_ratio Float 1.0 --
tracing -- -- The tracing options. Only effect when compiled with tokio-console feature.
tracing.tokio_console_addr String Unset The tokio console address.
memory -- -- The memory options.
memory.enable_heap_profiling Bool true Whether to enable heap profiling activation during startup.
When enabled, heap profiling will be activated if the MALLOC_CONF environment variable
is set to "prof:true,prof_active:false". The official image adds this env variable.
Default is true.

Datanode

Key Type Default Descriptions
node_id Integer Unset The datanode identifier and should be unique in the cluster.
default_column_prefix String Unset The default column prefix for auto-created time index, value, and native histogram columns.
Legacy OTLP summary columns keep their historical greptime_ prefix.
require_lease_before_startup Bool false Start services after regions have obtained leases.
It will block the datanode start if it can't receive leases in the heartbeat from metasrv.
init_regions_in_background Bool false Initialize all regions in the background during the startup.
By default, it provides services after all regions have been initialized.
init_regions_parallelism Integer 16 Parallelism of initializing regions.
max_concurrent_queries Integer 0 The maximum concurrent queries allowed to be executed. Zero means unlimited.
concurrent_query_limiter_timeout String 100ms Timeout to acquire a permit from the concurrent query limiter when max_concurrent_queries is reached.
enable_telemetry Bool true Enable telemetry to collect anonymous usage data. Enabled by default.
http -- -- The HTTP server options.
http.addr String 127.0.0.1:4000 The address to bind the HTTP server.
http.timeout String 0s HTTP request timeout. Set to 0 to disable timeout.
http.body_limit String 64MB HTTP request body limit.
The following units are supported: B, KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB.
Set to 0 to disable limit.
grpc -- -- The gRPC server options.
grpc.bind_addr String 127.0.0.1:3001 The address to bind the gRPC server.
grpc.server_addr String Unset The address advertised to the metasrv, and used for connections from outside the host.
If left empty or unset, the server will automatically use the IP address of the first network interface
on the host, with the same port number as the one specified in grpc.bind_addr.
grpc.runtime_size Integer 8 The number of server worker threads.
grpc.max_recv_message_size String 512MB The maximum receive message size for gRPC server.
grpc.max_send_message_size String 512MB The maximum send message size for gRPC server.
grpc.flight_compression String arrow_ipc Compression mode for datanode side Arrow IPC service. Available options:
- none: disable all compression
- transport: only enable gRPC transport compression (zstd)
- arrow_ipc: only enable Arrow IPC compression (lz4)
- all: enable all compression.
Default to arrow_ipc.
grpc.tls -- -- gRPC server TLS options, see mysql.tls section.
grpc.tls.mode String disable TLS mode.
grpc.tls.cert_path String Unset Certificate file path.
grpc.tls.key_path String Unset Private key file path.
grpc.tls.watch Bool false Watch for Certificate and key file change and auto reload.
For now, gRPC tls config does not support auto reload.
runtime -- -- The runtime options.
runtime.global_rt_size Integer Auto The number of threads to execute the runtime for global read operations.
Defaults to max(num_cpus, 2).
runtime.compact_rt_size Integer Auto The number of threads to execute compact operations.
Defaults to max(num_cpus / 2, 2).
runtime.compact_rt_max_blocking_threads Integer Auto The maximum number of blocking threads for compact operations.
Defaults to max(num_cpus / 2, 2).
runtime.query_rt_size Integer Auto The number of threads to execute datanode query operations.
Defaults to max(num_cpus - 1, 2).
runtime.ingest_rt_size Integer Auto The number of threads to execute datanode ingestion operations.
Defaults to max(num_cpus, 2).
runtime.experimental_workload_scheduler -- -- Experimental weighted, work-conserving query/write task scheduler.
runtime.experimental_workload_scheduler.enable Bool false Enable when concurrent queries and writes interfere with each other—for example, when long-running queries increase ingestion latency.
The weights set their relative runtime shares while both are backlogged. Disabled by default.
runtime.experimental_workload_scheduler.query_weight Integer 2 Relative query share while both query and write workloads are backlogged.
runtime.experimental_workload_scheduler.write_weight Integer 8 Relative write share while both query and write workloads are backlogged.
runtime.experimental_workload_scheduler.sample_every_polls Integer 16 Number of polls between scheduler fairness samples. Must be greater than zero.
meta_client -- -- The metasrv client options.
meta_client.metasrv_addrs Array -- The addresses of the metasrv.
meta_client.timeout String 3s Operation timeout.
meta_client.ddl_timeout String 10s DDL timeout.
meta_client.connect_timeout String 1s Connect server timeout.
meta_client.tcp_nodelay Bool true TCP_NODELAY option for accepted connections.
meta_client.metadata_cache_max_capacity Integer 100000 The configuration about the cache of the metadata.
meta_client.metadata_cache_ttl String 10m TTL of the metadata cache.
meta_client.metadata_cache_tti String 5m --
wal -- -- The WAL options.
wal.provider String raft_engine The provider of the WAL.
- raft_engine: the wal is stored in the local file system by raft-engine.
- kafka: it's remote wal that data is stored in Kafka.
- noop: it's a no-op WAL provider that does not store any WAL data.
Notes: any unflushed data will be lost when the datanode is shutdown.
- experimental_object_store: the wal is stored as objects in an object store.
Notes: experimental and not supported yet.
wal.dir String Unset The directory to store the WAL files.
It's only used when the provider is raft_engine.
wal.file_size String 128MB The size of the WAL segment file.
It's only used when the provider is raft_engine.
wal.purge_threshold String 1GB The threshold of the WAL size to trigger a purge.
It's only used when the provider is raft_engine.
wal.purge_interval String 1m The interval to trigger a purge.
It's only used when the provider is raft_engine.
wal.read_batch_size Integer 128 The read batch size.
It's only used when the provider is raft_engine.
wal.sync_write Bool false Whether to use sync write.
It's only used when the provider is raft_engine.
wal.enable_log_recycle Bool true Whether to reuse logically truncated log files.
It's only used when the provider is raft_engine.
wal.prefill_log_files Bool false Whether to pre-create log files on start up.
It's only used when the provider is raft_engine.
wal.sync_period String 5s Duration for fsyncing log files.
It's only used when the provider is raft_engine.
wal.recovery_parallelism Integer Auto Parallelism during WAL recovery. Defaults to the number of CPU cores; this example sets it to 2.
wal.broker_endpoints Array -- The Kafka broker endpoints.
It's only used when the provider is kafka.
wal.connect_timeout String 3s The connect timeout for kafka client.
It's only used when the provider is kafka.
wal.timeout String 5s The total request timeout for kafka client.
It's only used when the provider is kafka.
wal.max_batch_bytes String 1MB The max size of a single producer batch.
Warning: Kafka has a default limit of 1MB per message in a topic.
Defaults to 8MB when the provider is experimental_object_store.
It's only used when the provider is kafka or experimental_object_store.
wal.consumer_wait_timeout String 100ms The consumer wait timeout.
It's only used when the provider is kafka.
wal.create_index Bool false Whether to enable WAL index creation.
It's only used when the provider is kafka.
wal.dump_index_interval String 60s The interval for dumping WAL indexes.
It's only used when the provider is kafka.
wal.overwrite_entry_start_id Bool false Ignore missing entries during read WAL.
It's only used when the provider is kafka.

This option ensures that when Kafka messages are deleted, the system
can still successfully replay memtable data without throwing an
out-of-range error.
However, enabling this option might lead to unexpected data loss,
as the system will skip over missing entries instead of treating
them as critical errors.
wal.storage_provider String "" The name of the storage provider that holds the WAL objects, an empty name selects the default object store.
It's only used when the provider is experimental_object_store.
wal.prefix String wal The path prefix of the WAL objects inside the storage provider.
The objects are written under <prefix>/datanodes/<node_id>/epochs/<generation>, which is derived from this prefix.
It's only used when the provider is experimental_object_store.
wal.flush_interval String 100ms The interval of flushing buffered entries to the object store, at least 10ms, defaults to 100ms.
Each non-empty timer-triggered flush creates one object, and a batch that reaches max_batch_bytes is flushed immediately, so under sustained load the batch seals on size and the interval no longer matters.
When writes are sparse, a shorter interval lowers the acknowledgement latency of durable appends and raises the number of object requests: timer-triggered sealing creates at most one object per interval per node, and in enqueued mode the backlog thresholds can seal earlier. An enqueued append does not wait for its batch to seal once it is admitted, but the backlog thresholds can delay admission.
It's only used when the provider is experimental_object_store.
wal.ack_mode String durable When an append to the object store WAL returns.
- durable: an append returns after the object holding its entries is durable (the default).
- enqueued: an append returns once its entries are admitted and their ids are assigned, and the object is created in the background. A crash loses the unpersisted backlog.
It's only used when the provider is experimental_object_store.
wal.max_unpersisted_bytes String 64MB The size of the unpersisted backlog at which new appends stall until an upload completes, in enqueued mode.
Nothing is dropped and nothing is rejected; the threshold does not bound what a crash during an outage can lose.
It's only used when the provider is experimental_object_store and ack_mode is enqueued.
wal.max_unpersisted_age String 8s The age of the oldest unpersisted entry at which new appends stall until an upload completes, in enqueued mode.
It's only used when the provider is experimental_object_store and ack_mode is enqueued.
wal.on_corrupted_segment String skip What a read does with a segment that still does not decode after a second fetch, because its checksum does not match or its content disagrees with its footer entry.
- skip: the segment is skipped and recorded as a WAL hole of its region, a metric is incremented and a warning is logged; the other regions of the object are unaffected (the default).
- fail: the read fails, so the region does not open.
It's only used when the provider is experimental_object_store.
query -- -- The query engine options.
query.parallelism Integer 0 Parallelism of the query engine.
Default to 0, which means the number of CPU cores.
query.memory_pool_size String 0 Memory pool size for query execution operators (aggregation, sorting, join).
Supports absolute size (e.g., "2GB", "4GB") or percentage of system memory (e.g., "20%").
Setting it to 0 disables the limit. Defaults to 0 (unlimited); this example sets it to 50%.
When this limit is reached, queries will fail with ResourceExhausted error.
NOTE: This does NOT limit memory used by table scans.
query.experimental_memory_pool_policy String greedy Experimental memory pool allocation policy:
- "greedy" (default): first-come-first-served allocation; preserves current behavior.
- "fair": divides available memory among spillable operators and may spill earlier.
Only effective when memory_pool_size is bounded (>0).
query.experimental_spill_mode String default Spill mode:
- "default": preserve DataFusion built-in OS temp directory (default).
- "custom": explicitly configure spill path, quota, and compression.
- "disabled": explicitly disable disk spilling.
Set this to "custom" before using the path/quota/compression keys below.
query.experimental_spill_path String Unset Spill directory path. Ignored unless mode is "custom".
query.experimental_spill_max_temp_directory_size String 1GiB Maximum total size of spill directory (default: "1GiB").
Ignored unless mode is "custom".
query.experimental_spill_compression String uncompressed Compression for spilled data files: "uncompressed" (default), "lz4_frame", "zstd".
Ignored unless mode is "custom".
storage -- -- The data storage options.
storage.data_home String ./greptimedb_data The working home directory.
storage.type String File The storage type used to store the data.
- File: the data is stored in the local file system.
- S3: the data is stored in the S3 object storage.
- Gcs: the data is stored in the Google Cloud Storage.
- Azblob: the data is stored in the Azure Blob Storage.
- Oss: the data is stored in the Aliyun OSS.
- Hdfs: the data is stored in the Hadoop Distributed File System.
storage.bucket String Unset The S3 bucket name.
It's only used when the storage type is S3, Oss and Gcs.
storage.root String Unset The directory or object prefix under which data is stored.
It's only used when the storage type is S3, Oss, Gcs, Azblob and Hdfs.
storage.name_node String Unset The HDFS NameNode URI, for example, hdfs://127.0.0.1:9000.
It's only used when the storage type is Hdfs.
storage.options InlineTable Unset Additional options passed to the native HDFS client.
It's only used when the storage type is Hdfs.
storage.access_key_id String Unset The access key id of the aws account.
It's highly recommended to use AWS IAM roles instead of hardcoding the access key id and secret key.
It's only used when the storage type is S3 and Oss.
storage.secret_access_key String Unset The secret access key of the aws account.
It's highly recommended to use AWS IAM roles instead of hardcoding the access key id and secret key.
It's only used when the storage type is S3.
storage.access_key_secret String Unset The secret access key of the aliyun account.
It's only used when the storage type is Oss.
storage.account_name String Unset The account key of the azure account.
It's only used when the storage type is Azblob.
storage.account_key String Unset The account key of the azure account.
It's only used when the storage type is Azblob.
storage.scope String Unset The scope of the google cloud storage.
It's only used when the storage type is Gcs.
storage.credential_path String Unset The credential path of the google cloud storage.
It's only used when the storage type is Gcs.
storage.credential String Unset The credential of the google cloud storage.
It's only used when the storage type is Gcs.
storage.container String Unset The container of the azure account.
It's only used when the storage type is Azblob.
storage.sas_token String Unset The sas token of the azure account.
It's only used when the storage type is Azblob.
storage.endpoint String Unset The endpoint of the S3 service.
It's only used when the storage type is S3, Oss, Gcs and Azblob.
storage.region String Unset The region of the S3 service.
It's only used when the storage type is S3, Oss, Gcs and Azblob.
storage.http_client -- -- The http client options to the storage.
It's only used when the storage type is S3, Oss, Gcs and Azblob.
storage.http_client.pool_max_idle_per_host Integer 1024 The maximum idle connection per host allowed in the pool.
storage.http_client.connect_timeout String 30s The timeout for only the connect phase of a http client.
storage.http_client.timeout String 30s The total request timeout, applied from when the request starts connecting until the response body has finished.
Also considered a total deadline.
storage.http_client.pool_idle_timeout String 90s The timeout for idle sockets being kept-alive.
storage.http_client.skip_ssl_validation Bool false To skip the ssl verification
Security Notice: Setting skip_ssl_validation = true disables certificate verification, making connections vulnerable to man-in-the-middle attacks. Only use this in development or trusted private networks.
[[region_engine]] -- -- The region engine options. You can configure multiple region engines.
Each engine type (mito, file, metric) may appear only once; duplicates cause startup to fail.
region_engine.mito -- -- The Mito engine options.
region_engine.mito.num_workers Integer Auto Number of region workers. Defaults to half of the available CPU cores, rounded up.
region_engine.mito.worker_channel_size Integer 128 Request channel size of each worker.
region_engine.mito.worker_request_batch_size Integer 64 Max batch size for a worker to handle requests.
region_engine.mito.manifest_checkpoint_distance Integer 10 Number of meta action updated to trigger a new checkpoint for the manifest.
region_engine.mito.experimental_manifest_keep_removed_file_count Integer 256 Number of removed files to keep in manifest's removed_files field before also
remove them from removed_files. Mostly for debugging purpose.
If set to 0, it will only use keep_removed_file_ttl to decide when to remove files
from removed_files field.
region_engine.mito.experimental_manifest_keep_removed_file_ttl String 1h How long to keep removed files in the removed_files field of manifest
after they are removed from manifest.
files will only be removed from removed_files field
if both keep_removed_file_count and keep_removed_file_ttl is reached.
region_engine.mito.compress_manifest Bool false Whether to compress manifest and checkpoint file by gzip (default false).
region_engine.mito.experimental_enable_series_index Bool false Under development; do not enable. Whether to enable series indexes.
Indexes are stored on the local filesystem under {data_home}/series_index.
region_engine.mito.experimental_series_index_max_size String 5GiB Approximate series and range index size limit in open regions, shared across workers.
Workers periodically refresh usage and skip maintenance when full. In-flight reconciliation
can exceed the limit. Closed-region files, temporary output, catalogs, and old snapshots
retained by readers are not counted.
Minimum: 1KiB. Takes effect on restart.
region_engine.mito.experimental_enable_range_index Bool false Whether to build and query range indexes when series indexes are enabled.
Obsolete range-index metadata and files are still cleaned up when disabled.
region_engine.mito.experimental_series_index_maintenance_interval String 5m Interval between series-index maintenance runs. Zero uses the default of 5 min.
region_engine.mito.experimental_series_index_bucket_width String 5days Requested minimum series-index bucket width (default: 5 days), rounded up to
an exact multiple of each region's compaction time window.
region_engine.mito.max_background_flushes Integer Auto Max number of running background flush jobs (default: 1/2 of cpu cores).
region_engine.mito.max_background_compactions Integer Auto Max number of running background compaction jobs (default: 1/4 of cpu cores).
region_engine.mito.max_background_purges Integer Auto Max number of running background purge jobs (default: number of cpu cores).
region_engine.mito.experimental_compaction_memory_limit String 0 Memory budget for compaction tasks.
Supports absolute size (e.g., "2GiB", "512MB") or percentage of system memory (e.g., "50%").
Setting it to 0 or "unlimited" disables the limit.
region_engine.mito.experimental_compaction_on_exhausted String wait Behavior when compaction cannot acquire memory from the budget.
Options: "wait" (default, 10s), "wait()", "fail"
region_engine.mito.auto_flush_interval String 10m Interval to auto flush a region if it has not flushed yet.
region_engine.mito.global_write_buffer_size String Auto Global write buffer size for all regions. If not set, it's default to 1/8 of OS memory with a max limitation of 1GB.
region_engine.mito.global_write_buffer_reject_size String Auto Global write buffer size threshold to reject write requests. If not set, it's default to 2 times of global_write_buffer_size
region_engine.mito.default_region_write_buffer_size String 0 Default write buffer size for each region. Regions stall at this size and reject writes at twice this size. Setting it to 0 disables both limits unless the table specifies write_buffer_size.
region_engine.mito.sst_meta_cache_size String Auto Cache size for SST metadata. Setting it to 0 to disable the cache.
If not set, it's default to 1/8 of OS memory with a max limitation of 512MB.
region_engine.mito.vector_cache_size String Auto Cache size for vectors and arrow arrays. Setting it to 0 to disable the cache.
If not set, it's default to 1/16 of OS memory with a max limitation of 512MB.
region_engine.mito.page_cache_size String Auto Cache size for pages of SST row groups. Setting it to 0 to disable the cache.
If not set, it's default to 1/8 of OS memory.
region_engine.mito.selector_result_cache_size String Auto Cache size for time series selector (e.g. last_value()). Setting it to 0 to disable the cache.
If not set, it's default to 1/16 of OS memory with a max limitation of 512MB.
region_engine.mito.range_result_cache_size String Auto Cache size for flat range scan results. Setting it to 0 to disable the cache.
If not set, it's default to 1/16 of OS memory with a max limitation of 512MB.
region_engine.mito.prefilter_result_cache_size String Auto Cache size for prefilter results. Setting it to 0 to disable the cache.
If not set, it's default to 1/32 of OS memory with a max limitation of 128MB.
region_engine.mito.enable_write_cache Bool false Whether to enable the write cache, it's enabled by default when using object storage. It is recommended to enable it when using object storage for better performance.
region_engine.mito.write_cache_path String "" File system path for write cache, defaults to {data_home}.
region_engine.mito.write_cache_size String 5GiB Capacity for write cache. If your disk space is sufficient, it is recommended to set it larger.
region_engine.mito.write_cache_ttl String Unset TTL for write cache.
region_engine.mito.preload_index_cache Bool true Preload index (puffin) files into cache on region open (default: true).
When enabled, index files are loaded into the write cache during region initialization,
which can improve query performance at the cost of longer startup times.
region_engine.mito.index_cache_percent Integer 20 Percentage of write cache capacity allocated for index (puffin) files (default: 20).
The remaining capacity is used for data (parquet) files.
Must be between 0 and 100 (exclusive). For example, with a 5GiB write cache and 20% allocation,
1GiB is reserved for index files and 4GiB for data files.
region_engine.mito.enable_refill_cache_on_read Bool true Enable refilling cache on read operations (default: true).
When disabled, cache refilling on read won't happen.
region_engine.mito.manifest_cache_size String 256MB Capacity for manifest cache (default: 256MB).
region_engine.mito.sst_write_buffer_size String 8MB Buffer size for SST writing.
region_engine.mito.max_concurrent_scan_files Integer 384 Maximum number of SST files to scan concurrently.
region_engine.mito.allow_stale_entries Bool false Whether to allow stale WAL entries read during replay.
region_engine.mito.scan_memory_limit String unlimited Memory limit for table scans across all queries.
Supports absolute size (e.g., "2GB") or percentage of system memory (e.g., "20%").
Setting it to 0 or "unlimited" disables the limit.
region_engine.mito.scan_memory_on_exhausted String fail Controls what happens when a scan cannot get memory immediately.
"fail" (default) fails fast and is the recommended option for most users.
"wait" / "wait()" waits for memory to become available. This is mainly
for advanced tuning in bursty workloads where temporary contention is common and
higher latency is acceptable.
"wait" means "wait(10s)", not unlimited waiting.
region_engine.mito.min_compaction_interval String 0m Minimum time interval between two compactions.
To align with the old behavior, the default value is 0 (no restrictions).
region_engine.mito.schedule_compaction_after_edit Bool true Whether to allow to schedule a compaction after a successful region edit.

Setting this to "true" is a necessary but not sufficient condition for scheduling compaction after a region edit.
Other constraints, such as "min_compaction_interval", may still prevent compaction from being scheduled.
Setting this to "false", however, guarantees that compaction will not be scheduled after a region edit.
region_engine.mito.default_flat_format Bool true Whether to enable flat format as the default SST format.
region_engine.mito.experimental_series_scan_v2 Bool false Whether to enable the experimental two-phase mode for eligible metric series scans.
region_engine.mito.index -- -- The options for index in Mito engine.
region_engine.mito.index.aux_path String "" Auxiliary directory path for the index in filesystem, used to store intermediate files for
creating the index and staging files for searching the index, defaults to {data_home}/index_intermediate.
The default name for this directory is index_intermediate for backward compatibility.

This path contains two subdirectories:
- __intm: for storing intermediate files used during creating index.
- staging: for storing staging files used during searching index.
region_engine.mito.index.staging_size String 2GB The max capacity of the staging directory.
region_engine.mito.index.staging_ttl String 7d The TTL of the staging directory.
Defaults to 7 days.
Setting it to "0s" to disable TTL.
region_engine.mito.index.metadata_cache_size String 64MiB Cache size for inverted index metadata.
region_engine.mito.index.content_cache_size String 128MiB Cache size for inverted index content.
region_engine.mito.index.content_cache_page_size String 64KiB Page size for inverted index content cache.
region_engine.mito.index.result_cache_size String 128MiB Cache size for index result.
region_engine.mito.inverted_index -- -- The options for inverted index in Mito engine.
region_engine.mito.inverted_index.create_on_flush String auto Whether to create the index on flush.
- auto: automatically (default)
- disable: never
region_engine.mito.inverted_index.create_on_compaction String auto Whether to create the index on compaction.
- auto: automatically (default)
- disable: never
region_engine.mito.inverted_index.apply_on_query String auto Whether to apply the index on query
- auto: automatically (default)
- disable: never
region_engine.mito.inverted_index.mem_threshold_on_create String auto Memory threshold for performing an external sort during index creation.
- auto: automatically determine the threshold based on the system memory size (default)
- unlimited: no memory limit
- [size] e.g. 64MB: fixed memory threshold
region_engine.mito.inverted_index.intermediate_path String "" Deprecated, use region_engine.mito.index.aux_path instead.
region_engine.mito.fulltext_index -- -- The options for full-text index in Mito engine.
region_engine.mito.fulltext_index.create_on_flush String auto Whether to create the index on flush.
- auto: automatically (default)
- disable: never
region_engine.mito.fulltext_index.create_on_compaction String auto Whether to create the index on compaction.
- auto: automatically (default)
- disable: never
region_engine.mito.fulltext_index.apply_on_query String auto Whether to apply the index on query
- auto: automatically (default)
- disable: never
region_engine.mito.fulltext_index.mem_threshold_on_create String auto Memory threshold for index creation.
- auto: automatically determine the threshold based on the system memory size (default)
- unlimited: no memory limit
- [size] e.g. 64MB: fixed memory threshold
region_engine.mito.bloom_filter_index -- -- The options for bloom filter index in Mito engine.
region_engine.mito.bloom_filter_index.create_on_flush String auto Whether to create the index on flush.
- auto: automatically (default)
- disable: never
region_engine.mito.bloom_filter_index.create_on_compaction String auto Whether to create the index on compaction.
- auto: automatically (default)
- disable: never
region_engine.mito.bloom_filter_index.apply_on_query String auto Whether to apply the index on query
- auto: automatically (default)
- disable: never
region_engine.mito.bloom_filter_index.mem_threshold_on_create String auto Memory threshold for the index creation.
- auto: automatically determine the threshold based on the system memory size (default)
- unlimited: no memory limit
- [size] e.g. 64MB: fixed memory threshold
region_engine.mito.gc -- -- --
region_engine.mito.gc.enable Bool false Whether GC is enabled. Need to be the same with metasrv's gc.enable or unexpected behavior will occur
region_engine.mito.gc.lingering_time String 1h Lingering time before deleting files.
Should be long enough to allow long running queries to finish.
If set to None, then unused files will be deleted immediately.
region_engine.mito.gc.unknown_file_lingering_time String 1d Lingering time before deleting unknown files (files with undetermined expel time).
Only applies during full file listing GC.
This uses the object's last-modified timestamp as a heuristic (strict less-than comparison);
do not configure this value too small in production to avoid deleting pre-manifest files
from in-progress compaction or flush.
For active/open regions, an unknown file is deleted only if its object last-modified time exceeds this TTL.
If the object store does not provide a last-modified timestamp, the file is conservatively kept.
For dropped regions, unknown files are deleted immediately.
region_engine.file -- -- Enable the file engine.
region_engine.metric -- -- Metric engine options.
logging -- -- The logging options.
logging.dir String ./greptimedb_data/logs The directory to store the log files. If set to empty, logs will not be written to files.
logging.level String Unset The log level. Can be info/debug/warn/error.
logging.enable_otlp_tracing Bool false Enable OTLP tracing.
logging.otlp_endpoint String http://localhost:4318/v1/traces The OTLP tracing endpoint.
logging.append_stdout Bool true Whether to append logs to stdout.
logging.enable_file_logging Bool true Whether to write logs to files in dir.
logging.log_format String text The log format. Can be text/json.
logging.max_log_files Integer 720 The maximum amount of log files.
logging.max_log_dir_size String 0B The maximum total size of managed log files in dir.
Old closed log files are removed before writing when necessary. Active files may exceed it. Set to 0B to disable.
logging.otlp_export_protocol String http The OTLP tracing export protocol. Can be grpc/http.
logging.otlp_headers -- -- Additional OTLP headers, only valid when using OTLP http
logging.tracing_sample_ratio -- Unset The percentage of tracing will be sampled and exported.
Valid range [0, 1], 1 means all traces are sampled, 0 means all traces are not sampled, the default value is 1.
ratio > 1 are treated as 1. Fractions < 0 are treated as 0
logging.tracing_sample_ratio.default_ratio Float 1.0 --
tracing -- -- The tracing options. Only effect when compiled with tokio-console feature.
tracing.tokio_console_addr String Unset The tokio console address.
memory -- -- The memory options.
memory.enable_heap_profiling Bool true Whether to enable heap profiling activation during startup.
When enabled, heap profiling will be activated if the MALLOC_CONF environment variable
is set to "prof:true,prof_active:false". The official image adds this env variable.
Default is true.

Flownode

Key Type Default Descriptions
node_id Integer Unset The flownode identifier and should be unique in the cluster.
flow -- -- flow engine options.
flow.batching_mode -- -- --
flow.batching_mode.query_timeout String 600s The default batching engine query timeout is 10 minutes.
flow.batching_mode.slow_query_threshold String 60s will output a warn log for any query that runs for more that this threshold
flow.batching_mode.experimental_min_refresh_duration String 5s The minimum duration between two queries execution by batching mode task
flow.batching_mode.grpc_conn_timeout String 5s The gRPC connection timeout
flow.batching_mode.experimental_grpc_max_retries Integer 3 The gRPC max retry number
flow.batching_mode.experimental_frontend_scan_timeout String 30s Flow wait for available frontend timeout,
if failed to find available frontend after frontend_scan_timeout elapsed, return error
which prevent flownode from starting
flow.batching_mode.experimental_max_filter_num_per_query Integer 20 Maximum number of filters allowed in a single query
flow.batching_mode.experimental_time_window_merge_threshold Integer 3 Time window merge distance
flow.batching_mode.experimental_enable_incremental_read Bool false Whether to enable experimental flow incremental source reads.
When disabled, batching flows always execute full-snapshot queries.
Can be overridden per flow with WITH (experimental_enable_incremental_read = 'true').
flow.batching_mode.read_preference String Leader Read preference of the Frontend client.
flow.batching_mode.frontend_tls -- -- --
flow.batching_mode.frontend_tls.enabled Bool false Whether to enable TLS for client.
flow.batching_mode.frontend_tls.server_ca_cert_path String Unset Server Certificate file path.
flow.batching_mode.frontend_tls.client_cert_path String Unset Client Certificate file path.
flow.batching_mode.frontend_tls.client_key_path String Unset Client Private key file path.
grpc -- -- The gRPC server options.
grpc.bind_addr String 127.0.0.1:3004 The address to bind the gRPC server. Defaults to 127.0.0.1:3004; this example sets it to 127.0.0.1:6800.
grpc.server_addr String Unset The address advertised to the metasrv, and used for connections from outside the host.
If left empty or unset, the server will automatically use the IP address of the first network interface
on the host, with the same port number as the one specified in grpc.bind_addr.
grpc.runtime_size Integer 8 The number of server worker threads. Defaults to 8; this example sets it to 2.
grpc.max_recv_message_size String 512MB The maximum receive message size for gRPC server.
grpc.max_send_message_size String 512MB The maximum send message size for gRPC server.
http -- -- The HTTP server options.
http.addr String 127.0.0.1:4000 The address to bind the HTTP server.
http.timeout String 0s HTTP request timeout. Set to 0 to disable timeout.
http.body_limit String 64MB HTTP request body limit.
The following units are supported: B, KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB.
Set to 0 to disable limit.
meta_client -- -- The metasrv client options.
meta_client.metasrv_addrs Array -- The addresses of the metasrv.
meta_client.timeout String 3s Operation timeout.
meta_client.ddl_timeout String 10s DDL timeout.
meta_client.connect_timeout String 1s Connect server timeout.
meta_client.tcp_nodelay Bool true TCP_NODELAY option for accepted connections.
meta_client.metadata_cache_max_capacity Integer 100000 The configuration about the cache of the metadata.
meta_client.metadata_cache_ttl String 10m TTL of the metadata cache.
meta_client.metadata_cache_tti String 5m --
logging -- -- The logging options.
logging.dir String ./greptimedb_data/logs The directory to store the log files. If set to empty, logs will not be written to files.
logging.level String Unset The log level. Can be info/debug/warn/error.
logging.enable_otlp_tracing Bool false Enable OTLP tracing.
logging.otlp_endpoint String http://localhost:4318/v1/traces The OTLP tracing endpoint.
logging.append_stdout Bool true Whether to append logs to stdout.
logging.enable_file_logging Bool true Whether to write logs to files in dir.
logging.log_format String text The log format. Can be text/json.
logging.max_log_files Integer 720 The maximum amount of log files.
logging.max_log_dir_size String 0B The maximum total size of managed log files in dir.
Old closed log files are removed before writing when necessary. Active files may exceed it. Set to 0B to disable.
logging.otlp_export_protocol String http The OTLP tracing export protocol. Can be grpc/http.
logging.otlp_headers -- -- Additional OTLP headers, only valid when using OTLP http
logging.tracing_sample_ratio -- Unset The percentage of tracing will be sampled and exported.
Valid range [0, 1], 1 means all traces are sampled, 0 means all traces are not sampled, the default value is 1.
ratio > 1 are treated as 1. Fractions < 0 are treated as 0
logging.tracing_sample_ratio.default_ratio Float 1.0 --
tracing -- -- The tracing options. Only effect when compiled with tokio-console feature.
tracing.tokio_console_addr String Unset The tokio console address.
query -- -- --
query.parallelism Integer 1 Parallelism of the query engine for query sent by flownode.
Default to 1, so it won't use too much cpu or memory
query.memory_pool_size String 0 Memory pool size for query execution operators (aggregation, sorting, join).
Supports absolute size (e.g., "1GB", "2GB") or percentage of system memory (e.g., "20%").
Setting it to 0 disables the limit. Defaults to 0 (unlimited); this example sets it to 50%.
When this limit is reached, queries will fail with ResourceExhausted error.
NOTE: This does NOT limit memory used by table scans.
query.experimental_memory_pool_policy String greedy Experimental memory pool allocation policy:
- "greedy" (default): first-come-first-served allocation; preserves current behavior.
- "fair": divides available memory among spillable operators and may spill earlier.
Only effective when memory_pool_size is bounded (>0).
query.experimental_spill_mode String default Spill mode:
- "default": preserve DataFusion built-in OS temp directory (default).
- "custom": explicitly configure spill path, quota, and compression.
- "disabled": explicitly disable disk spilling.
Set this to "custom" before using the path/quota/compression keys below.
query.experimental_spill_path String Unset Spill directory path. Ignored unless mode is "custom".
query.experimental_spill_max_temp_directory_size String 1GiB Maximum total size of spill directory (default: "1GiB").
Ignored unless mode is "custom".
query.experimental_spill_compression String uncompressed Compression for spilled data files: "uncompressed" (default), "lz4_frame", "zstd".
Ignored unless mode is "custom".
memory -- -- The memory options.
memory.enable_heap_profiling Bool true Whether to enable heap profiling activation during startup.
When enabled, heap profiling will be activated if the MALLOC_CONF environment variable
is set to "prof:true,prof_active:false". The official image adds this env variable.
Default is true.