Files
greptimedb/config/standalone.example.toml
T
jeremyhi deae75ffee fix(log-store): refuse to open object store WAL regions whose history is missing (#9561)
* fix(log-store): refuse to open object store WAL regions whose history is missing

A region persists only its WAL prefix, not the object store that holds it.
After `[wal].storage_provider`, or the bucket or root of the provider it
names, changes, or after the objects under the prefix are deleted, every
region reopened against a prefix without its history, replayed nothing and
silently lost its acknowledged writes that were not flushed.

Add `LogStore::check_region_history` and `LogStore::start_region_history`,
whose defaults accept and do nothing, so Raft Engine and Kafka are
unchanged. The object store WAL reports the history of a region as missing
when recovery found no object under the prefix, or when the region's
watermark names an object above the newest object holding entries that
recovery indexed, unless the catalog holds the region's entries through the
watermark or the region's history starts in the store. Both rest on
invariants of the prefix: an opened prefix is never empty, and a persisted
watermark names an object at or below the newest object holding entries,
which collection never deletes. The segment of release records counts as
entries, since an object holding only release records can be the one
collection keeps; start objects hold no segment and do not move the bound.

When a store that found its prefix empty refuses a region, the actor gives
up its start object, so the next start refuses the region again. It agrees
only while nothing was handed an id, no batch, append, release or create
waits, no failed release is kept, no other object was listed, the catalog
holds only the start object and no region's history started in the store,
deciding under the lock region registration takes. It then records a
terminal error, deletes the object as one of its tracked deletes, which
`stop` waits for, and answers every refusal once that delete settled.

Mito runs the check before it replays an existing region on open, on
catch-up and when a create finds the region present, with the larger of
the flushed and truncated entry ids, and records the history of a region
it creates; regions that skip the WAL and noop regions are not checked. A
missing history fails the open with an error naming the region, the
prefix, the configured storage and how to move the WAL by copying its
objects. The new `[wal].accept_missing_history` option, off by default,
opens such a region without its unflushed writes instead, with a warning
and the counter
`greptime_logstore_object_store_wal_missing_history_accepted_total`.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* docs(rfc): describe the object store WAL history check

Describe the open-time history check, the prefix invariants it relies on,
how release records bound the history, the start object withdrawal of a
refused open and the window that remains, the accept_missing_history
option and how to move a WAL by copying its objects, and update the
metrics, Mito integration, evidence, compatibility and cluster alignment
sections.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(log-store): unindex the start object a refused open withdrew

A withdrawal that deleted the start object left it in the catalog and in the indexed object and byte gauges for the rest of the process. Remove it from both once the delete succeeded, and keep it when the delete failed.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* docs(rfc): separate the withdrawal windows of start objects and entries

An append or a release admitted before a refusal leaves an object holding entries, which may have collected the start object, so the next start also passes watermarks at or below that object.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(log-store): keep a successor's chain complete across a late withdrawal

A start object withdrawn after another instance recovered it was linked by that instance's start object, which carried reclaim boundary 0, so once the delete landed the next open found no complete chain. A start object now reclaims every object below it when the recovered chain holds no segment, so such a delete stays below its boundary.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix(mito2): record a created region's WAL history before its manifest

A create recorded the region's WAL history after it persisted the manifest, so a withdrawal in between failed the create but left a manifest that the next start refused as a region without its history. Record the history first, so such a create fails before it writes anything.

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* docs(rfc): describe the start boundary of a chain without segments and the create order

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2026-10-15 12:38:01 +00:00

1132 lines
44 KiB
TOML

## The default timezone of the server.
## @toml2docs:none-default
default_timezone = "UTC"
## The default column prefix for auto-created time index, value, and native histogram columns.
## Legacy OTLP summary columns keep their historical `greptime_` prefix.
## @toml2docs:none-default
default_column_prefix = "greptime"
## Server-side global switch for auto table creation on write.
## When `false`, a missing table is never auto-created even if the request sets the `auto_create_table` hint to `true`. Default: `true`.
#+ auto_create_table = true
## The user provider for authentication.
## Examples: "static_user_provider:file:/path/to/users", "static_user_provider:cmd:greptime_user=greptime_pwd"
## Password verifier formats: "plain:<password>", "pbkdf2_sha256:<iterations>:<hex_salt>:<hex_hash>",
## "mysql_native_password:<hex_sha1_sha1_password>",
## "pg_scram_sha256:<iterations>:<hex_salt>:<hex_stored_key>:<hex_server_key>"
## "pbkdf2_sha256" and "pg_scram_sha256" protect passwords at rest, but such users cannot log in over MySQL:
## `static_user_provider` and `watch_file_user_provider` only negotiate `mysql_native_password`, which cannot verify these formats.
## "mysql_native_password" is MySQL-specific and cannot authenticate over PostgreSQL at all.
## PostgreSQL SCRAM only covers "plain" and "pg_scram_sha256" users; if any user is "pbkdf2_sha256" or
## "mysql_native_password", PostgreSQL falls back to cleartext password auth for every user.
## For "pg_scram_sha256" users, keep the default iteration count (4096) and salt length (16): both are
## observable in the SCRAM server-first message, and non-default values weaken resistance to username
## enumeration.
## @toml2docs:none-default
#+ user_provider = "static_user_provider:file:/path/to/users"
## Maximum total memory for all concurrent write request bodies and messages (HTTP, gRPC, Flight).
## Set to 0 to disable the limit. Default: "0" (unlimited)
## @toml2docs:none-default="0"
#+ max_in_flight_write_bytes = "1GB"
## Policy when write bytes quota is exhausted.
## Options: "wait" (default, 10s timeout), "wait(<duration>)" (e.g., "wait(30s)"), "fail"
#+ write_bytes_exhausted_policy = "wait"
## Initialize all regions in the background during the startup.
## By default, it provides services after all regions have been initialized.
init_regions_in_background = false
## Parallelism of initializing regions.
init_regions_parallelism = 16
## Enable telemetry to collect anonymous usage data. Enabled by default.
#+ enable_telemetry = true
## The runtime options.
#+ [runtime]
## The number of threads to execute the runtime for global read operations.
## Defaults to max(num_cpus, 2).
## @toml2docs:none-default="Auto"
#+ global_rt_size = 8
## The number of threads to execute compact operations.
## Defaults to max(num_cpus / 2, 2).
## @toml2docs:none-default="Auto"
#+ compact_rt_size = 4
## The maximum number of blocking threads for compact operations.
## Defaults to max(num_cpus / 2, 2).
## @toml2docs:none-default="Auto"
#+ compact_rt_max_blocking_threads = 4
## Experimental weighted, work-conserving query/write task scheduler.
#+ [runtime.experimental_workload_scheduler]
## Enable when concurrent queries and writes interfere with each other—for example, when long-running queries increase ingestion latency.
## The weights set their relative runtime shares while both are backlogged. Disabled by default.
#+ enable = false
## Relative query share while both query and write workloads are backlogged.
#+ query_weight = 2
## Relative write share while both query and write workloads are backlogged.
#+ write_weight = 8
## Number of polls between scheduler fairness samples. Must be greater than zero.
#+ sample_every_polls = 16
## The HTTP server options.
[http]
## The address to bind the HTTP server.
addr = "127.0.0.1:4000"
## HTTP request timeout. Set to 0 to disable timeout.
## When synchronous Prometheus, OTLP metrics, or ordinary-table batching is enabled, a nonzero timeout is
## raised to at least the largest active flush interval plus 1 second. The intervals come from
## `pending_rows_batcher.logical_table.pending_rows_flush_interval` and `pending_rows_batcher.pending_rows_flush_interval`.
timeout = "0s"
## HTTP request body limit.
## The following units are supported: `B`, `KB`, `KiB`, `MB`, `MiB`, `GB`, `GiB`, `TB`, `TiB`, `PB`, `PiB`.
## Set to 0 to disable limit.
body_limit = "64MB"
## HTTP CORS support, it's turned on by default
## This allows browser to access http APIs without CORS restrictions
enable_cors = true
## Customize allowed origins for HTTP CORS.
## @toml2docs:none-default
cors_allowed_origins = ["https://example.com"]
## Whether to start the dedicated public HTTP **API** server. This server serves
## only the `v1` interfaces plus the dashboard, and shares every other `[http]`
## option with the main server. It is disabled by default; set to `true` to enable.
enable_api_server = false
## The address to bind the dedicated HTTP API server, in the same form as `addr`.
## Defaults to `127.0.0.1:4006`.
api_server_addr = "127.0.0.1:4006"
## The gRPC server options.
[grpc]
## The address to bind the gRPC server.
bind_addr = "127.0.0.1:4001"
## The number of server worker threads.
runtime_size = 8
## Enable CORS for gRPC-Web clients in browsers. Disabled by default.
#+ enable_cors = false
## Origins allowed by gRPC CORS. An empty list allows any origin.
## @toml2docs:none-default
#+ cors_allowed_origins = ["https://example.com"]
## The maximum connection age for gRPC connection.
## The value can be a human-readable time string. For example: `10m` for ten minutes or `1h` for one hour.
## Refer to https://grpc.io/docs/guides/keepalive/ for more details.
## @toml2docs:none-default
#+ max_connection_age = "10m"
## gRPC server TLS options, see `mysql.tls` section.
[grpc.tls]
## TLS mode.
mode = "disable"
## Certificate file path.
## @toml2docs:none-default
cert_path = ""
## Private key file path.
## @toml2docs:none-default
key_path = ""
## Watch for Certificate and key file change and auto reload.
## For now, gRPC tls config does not support auto reload.
watch = false
## MySQL server options.
[mysql]
## Whether to enable.
enable = true
## The addr to bind the MySQL server.
addr = "127.0.0.1:4002"
## The number of server worker threads.
runtime_size = 2
## Server-side keep-alive time.
## Set to 0 (default) to disable.
keep_alive = "0s"
## Maximum entries in the MySQL prepared statement cache; default is 10,000.
prepared_stmt_cache_size= 10000
# MySQL server TLS options.
[mysql.tls]
## TLS mode, refer to https://www.postgresql.org/docs/current/libpq-ssl.html
## - `disable` (default value)
## - `prefer`
## - `require`
## - `verify-ca`
## - `verify-full`
mode = "disable"
## Certificate file path.
## @toml2docs:none-default
cert_path = ""
## Private key file path.
## @toml2docs:none-default
key_path = ""
## Watch for Certificate and key file change and auto reload
watch = false
## PostgresSQL server options.
[postgres]
## Whether to enable
enable = true
## The addr to bind the PostgresSQL server.
addr = "127.0.0.1:4003"
## The number of server worker threads.
runtime_size = 2
## Server-side keep-alive time.
## Set to 0 (default) to disable.
keep_alive = "0s"
## PostgresSQL server TLS options, see `mysql.tls` section.
[postgres.tls]
## TLS mode.
mode = "disable"
## Certificate file path.
## @toml2docs:none-default
cert_path = ""
## Private key file path.
## @toml2docs:none-default
key_path = ""
## Watch for Certificate and key file change and auto reload
watch = false
## OpenTSDB protocol options.
[opentsdb]
## Whether to enable OpenTSDB put in HTTP API.
enable = true
## InfluxDB protocol options.
[influxdb]
## Whether to enable InfluxDB protocol in HTTP API.
enable = true
## Default merge mode for tables automatically created by InfluxDB protocol.
## Available values: "last_non_null", "last_row".
default_merge_mode = "last_non_null"
## Ordinary-table batching for opted-in ingestion protocols.
## PENDING_ROWS_BATCH_SYNC defaults to true for both batchers. Set it to false to acknowledge
## queue admission without waiting for storage; later failures cannot be returned to the client.
## Omitted or empty protocols disables batching. Prom without metric engine uses this batcher.
## OTLP logs, traces and ordinary metrics use this batcher.
## MySQL and PostgreSQL use the same acknowledgement policy. Protocol timeouts are unchanged.
## Single-connection writes and INSERT SELECT may incur additional flush waits.
[pending_rows_batcher]
# protocols = [
# "influxdb",
# "opentsdb",
# "otlp",
# "logs",
# "loki",
# "splunk",
# "elasticsearch",
# "http_sql",
# "mysql",
# "postgres",
# "prom",
# ]
## Flush interval measured from the first pending submission. Zero disables batching.
pending_rows_flush_interval = "0s"
## Flush after a complete submission reaches this row threshold.
max_batch_rows = 100000
## Maximum concurrent flushes shared by the frontend batcher.
max_concurrent_flushes = 256
## Maximum queued submissions per table worker.
worker_channel_capacity = 65536
## Maximum admitted original requests awaiting completion.
max_inflight_requests = 3000
## Maximum number of queued table Flow notifications.
flow_notification_queue_capacity = 1024
## Metric-engine logical-table batching for Prom remote write and non-legacy OTLP metrics.
## Requires prom_store.with_metric_engine. Logs, traces and legacy metrics are not eligible.
## Enable independently with protocols and a nonzero flush interval.
## Omitted fields use independent defaults, not parent settings.
## Empty protocols or a zero interval disables logical batching without fallback.
## Omitting this entire section preserves legacy Prom batching; it does not enable OTLP batching.
[pending_rows_batcher.logical_table]
# protocols = ["prom", "otlp"]
## Flush interval measured from the first pending submission. Zero disables batching.
pending_rows_flush_interval = "0s"
## Flush after a complete submission reaches this row threshold.
max_batch_rows = 100000
## Maximum concurrent flushes shared by Prom and OTLP metrics.
max_concurrent_flushes = 256
## Maximum queued submissions per physical-table worker.
worker_channel_capacity = 65536
## Maximum admitted original requests awaiting completion.
max_inflight_requests = 3000
## Maximum number of queued logical-table Flow notifications.
flow_notification_queue_capacity = 1024
## Jaeger protocol options.
[jaeger]
## Whether to enable Jaeger protocol in HTTP API.
enable = true
## OpenTelemetry protocol options.
[otlp]
## Whether to enable OpenTelemetry protocol in HTTP API.
enable = true
## Maximum spans per trace ingest chunk. Set to 0 to disable splitting.
trace_ingest_chunk_size = 512
## Estimated memory budget for cached trace service/operation keys per frontend,
## shared across all catalogs, schemas, and trace tables. Set to 0 to disable caching.
## Defaults to 1/128 of the host or pod memory limit, with a minimum of 32MiB.
## Uses 32MiB if memory detection is unavailable.
## Excludes cache and allocator overhead; shared table names are charged per entry.
## @toml2docs:none-default="Auto"
#+ trace_aux_cache_size = "32MiB"
## Whether to synthesize the `greptime_otel_resource_info` table from OTLP metric
## resource attributes, so metrics-only services reach the semantic graph.
experimental_enable_resource_info = false
## Prometheus remote storage options
[prom_store]
## Whether to enable Prometheus remote write and read in HTTP API.
enable = true
## Whether to store the data from Prometheus remote write in metric engine.
with_metric_engine = true
## Whether to enable validation for Prometheus remote write requests.
## Available options:
## - strict: deny invalid UTF-8 strings (default).
## - lossy: allow invalid UTF-8 strings, replace invalid characters with REPLACEMENT_CHARACTER(U+FFFD).
## - unchecked: do not valid strings.
prom_validation_mode = "strict"
## The WAL options.
[wal]
## The provider of the WAL.
## - `raft_engine`: the wal is stored in the local file system by raft-engine.
## - `kafka`: it's remote wal that data is stored in Kafka.
## - `experimental_object_store`: the wal is stored as objects in an object store.<br/>**Notes: experimental.**
provider = "raft_engine"
## The directory to store the WAL files.
## **It's only used when the provider is `raft_engine`**.
## @toml2docs:none-default
dir = "./greptimedb_data/wal"
## The size of the WAL segment file.
## **It's only used when the provider is `raft_engine`**.
file_size = "128MB"
## The threshold of the WAL size to trigger a purge.
## **It's only used when the provider is `raft_engine`**.
purge_threshold = "1GB"
## The interval to trigger a purge.
## **It's only used when the provider is `raft_engine`**.
purge_interval = "1m"
## The read batch size.
## **It's only used when the provider is `raft_engine`**.
read_batch_size = 128
## Whether to use sync write.
## **It's only used when the provider is `raft_engine`**.
sync_write = false
## Whether to reuse logically truncated log files.
## **It's only used when the provider is `raft_engine`**.
enable_log_recycle = true
## Whether to pre-create log files on start up.
## **It's only used when the provider is `raft_engine`**.
prefill_log_files = false
## Duration for fsyncing log files.
## **It's only used when the provider is `raft_engine`**.
sync_period = "5s"
## Parallelism during WAL recovery. Defaults to the number of CPU cores; this example sets it to 2.
## @toml2docs:none-default="Auto"
recovery_parallelism = 2
## The Kafka broker endpoints.
## **It's only used when the provider is `kafka`**.
broker_endpoints = ["127.0.0.1:9092"]
## The connect timeout for kafka client.
## **It's only used when the provider is `kafka`**.
#+ connect_timeout = "3s"
## The total request timeout for kafka client.
## **It's only used when the provider is `kafka`**.
#+ timeout = "5s"
## Automatically create topics for WAL.
## Set to `true` to automatically create topics for WAL.
## Otherwise, use topics named `topic_name_prefix_[0..num_topics)`
auto_create_topics = true
## Number of topics.
## **It's only used when the provider is `kafka`**.
num_topics = 64
## Topic selector type.
## Available selector types:
## - `round_robin` (default)
## **It's only used when the provider is `kafka`**.
selector_type = "round_robin"
## A Kafka topic is constructed by concatenating `topic_name_prefix` and `topic_id`.
## i.g., greptimedb_wal_topic_0, greptimedb_wal_topic_1.
## **It's only used when the provider is `kafka`**.
topic_name_prefix = "greptimedb_wal_topic"
## Expected number of replicas of each partition.
## **It's only used when the provider is `kafka`**.
replication_factor = 1
## Above which a topic creation operation will be cancelled.
## **It's only used when the provider is `kafka`**.
create_topic_timeout = "30s"
## The max size of a single producer batch.
## Warning: Kafka has a default limit of 1MB per message in a topic.
## Defaults to `8MB` when the provider is `experimental_object_store`.
## **It's only used when the provider is `kafka` or `experimental_object_store`**.
max_batch_bytes = "1MB"
## The consumer wait timeout.
## **It's only used when the provider is `kafka`**.
consumer_wait_timeout = "100ms"
## Ignore missing entries during read WAL.
## **It's only used when the provider is `kafka`**.
##
## This option ensures that when Kafka messages are deleted, the system
## can still successfully replay memtable data without throwing an
## out-of-range error.
## However, enabling this option might lead to unexpected data loss,
## as the system will skip over missing entries instead of treating
## them as critical errors.
overwrite_entry_start_id = false
## The name of the storage provider that holds the WAL objects, an empty name selects the default object store.
## **It's only used when the provider is `experimental_object_store`**.
#+ storage_provider = ""
## The path prefix of the WAL objects inside the storage provider.
## The objects are written under `<prefix>/datanodes/<node_id>/epochs/<generation>`, which is derived from this prefix.
## **It's only used when the provider is `experimental_object_store`**.
#+ prefix = "wal"
## The interval of flushing buffered entries to the object store, at least `10ms`, defaults to `100ms`.
## Each non-empty timer-triggered flush creates one object, and a batch that reaches `max_batch_bytes` is flushed immediately, so the batch seals on size only when the admitted bytes reach `max_batch_bytes` before the next timer tick; otherwise it seals on the timer, whatever the load.
## When writes are sparse, a shorter interval lowers the acknowledgement latency of `durable` appends and raises the number of object requests: timer-triggered sealing creates at most one object per interval per node, and in `enqueued` mode the backlog thresholds can seal earlier. An `enqueued` append does not wait for its batch to seal once it is admitted, but the backlog thresholds can delay admission.
## **It's only used when the provider is `experimental_object_store`**.
#+ flush_interval = "100ms"
## When an append to the object store WAL returns.
## - `durable`: an append returns after the object holding its entries is durable (the default).
## - `enqueued`: an append returns once its entries are admitted and their ids are assigned, and the object is created in the background. A crash loses the unpersisted backlog.
## **It's only used when the provider is `experimental_object_store`**.
#+ ack_mode = "durable"
## The size of the unpersisted backlog at which new appends stall until an upload completes, in `enqueued` mode.
## Nothing is dropped and nothing is rejected; the threshold does not bound what a crash during an outage can lose.
## **It's only used when the provider is `experimental_object_store` and `ack_mode` is `enqueued`**.
#+ max_unpersisted_bytes = "64MB"
## The age of the oldest unpersisted entry at which new appends stall until an upload completes, in `enqueued` mode.
## **It's only used when the provider is `experimental_object_store` and `ack_mode` is `enqueued`**.
#+ max_unpersisted_age = "8s"
## What a read does with a segment that still does not decode after a second fetch, because its checksum does not match or its content disagrees with its footer entry.
## - `skip`: the segment is skipped and recorded as a WAL hole of its region, a metric is incremented and a warning is logged; the other regions of the object are unaffected (the default).
## - `fail`: the read fails, so the region does not open.
## **It's only used when the provider is `experimental_object_store`**.
#+ on_corrupted_segment = "skip"
## Whether a region opens although the WAL objects under the prefix do not hold its history.
## An existing region fails to open when the prefix held no object when the WAL opened, or when the flushed entry id of the region names an object above every object of the prefix that holds entries. Both happen after `storage_provider`, or the bucket or root of the provider it names, changed without the WAL objects being copied, or after the objects were deleted.
## To move the WAL, stop the instance, copy every object under the prefix to the new location and then change the configuration; the regions open with all their writes.
## When the old WAL is lost, set this to `true` for one start: the regions open without their writes that were not flushed, which are lost, a warning names each region, and `greptime_logstore_object_store_wal_missing_history_accepted_total` counts them. Turn it off again once the instance has written to the WAL after that start; until then a later start refuses a region whose flushed entry id names an object above every object of the prefix that holds entries.
## **It's only used when the provider is `experimental_object_store`**.
#+ accept_missing_history = false
# The Kafka SASL configuration.
# **It's only used when the provider is `kafka`**.
# Available SASL mechanisms:
# - `PLAIN`
# - `SCRAM-SHA-256`
# - `SCRAM-SHA-512`
# [wal.sasl]
# type = "SCRAM-SHA-512"
# username = "user_kafka"
# password = "secret"
# The Kafka TLS configuration.
# **It's only used when the provider is `kafka`**.
# [wal.tls]
# server_ca_cert_path = "/path/to/server_cert"
# client_cert_path = "/path/to/client_cert"
# client_key_path = "/path/to/key"
## Metadata storage options.
[metadata_store]
## The size of the metadata store log file.
file_size = "64MB"
## The threshold of the metadata store size to trigger a purge.
purge_threshold = "256MB"
## The interval of the metadata store to trigger a purge.
purge_interval = "1m"
## Procedure storage options.
[procedure]
## Procedure max retry time.
max_retry_times = 3
## Initial retry delay of procedures, increases exponentially
retry_delay = "500ms"
## Max running procedures.
## The maximum number of procedures that can be running at the same time.
## If the number of running procedures exceeds this limit, the procedure will be rejected.
max_running_procedures = 128
# Example of using S3 as the storage.
# [storage]
# type = "S3"
# bucket = "greptimedb"
# root = "data"
# access_key_id = "test"
# secret_access_key = "123456"
# endpoint = "https://s3.amazonaws.com"
# region = "us-west-2"
# enable_virtual_host_style = false
# disable_ec2_metadata = false
# Example of using Oss as the storage.
# [storage]
# type = "Oss"
# bucket = "greptimedb"
# root = "data"
# access_key_id = "test"
# access_key_secret = "123456"
# endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
# Example of using Azblob as the storage.
# [storage]
# type = "Azblob"
# container = "greptimedb"
# root = "data"
# account_name = "test"
# account_key = "123456"
# endpoint = "https://greptimedb.blob.core.windows.net"
# sas_token = ""
# Example of using Gcs as the storage.
# [storage]
# type = "Gcs"
# bucket = "greptimedb"
# root = "data"
# scope = "test"
# credential_path = "123456"
# credential = "base64-credential"
# endpoint = "https://storage.googleapis.com"
# Example of using HDFS as the storage with the native Rust client.
# [storage]
# type = "Hdfs"
# root = "/greptimedb"
# name_node = "hdfs://127.0.0.1:9000"
# options = { "dfs.client.block.write.replace-datanode-on-failure.enable" = "true" }
## The query engine options.
[query]
## Parallelism of the query engine.
## Default to 0, which means the number of CPU cores.
parallelism = 0
## Memory pool size for query execution operators (aggregation, sorting, join).
## Supports absolute size (e.g., "2GB", "4GB") or percentage of system memory (e.g., "20%").
## Setting it to 0 disables the limit. Defaults to 0 (unlimited); this example sets it to `50%`.
## When this limit is reached, queries will fail with ResourceExhausted error.
## NOTE: This does NOT limit memory used by table scans.
## @toml2docs:none-default="0"
memory_pool_size = "50%"
## Experimental memory pool allocation policy:
## - "greedy" (default): first-come-first-served allocation; preserves current behavior.
## - "fair": divides available memory among spillable operators and may spill earlier.
## Only effective when `memory_pool_size` is bounded (>0).
#+ experimental_memory_pool_policy = "greedy"
# --- Experimental: DataFusion spill-to-disk controls ---
## Spill mode:
## - "default": preserve DataFusion built-in OS temp directory (default).
## - "custom": explicitly configure spill path, quota, and compression.
## - "disabled": explicitly disable disk spilling.
## Set this to "custom" before using the path/quota/compression keys below.
#+ experimental_spill_mode = "default"
## Spill directory path. Ignored unless mode is "custom".
## @toml2docs:none-default
#+ experimental_spill_path = "/path/to/spill"
## Maximum total size of spill directory (default: "1GiB").
## Ignored unless mode is "custom".
#+ experimental_spill_max_temp_directory_size = "1GiB"
## Compression for spilled data files: "uncompressed" (default), "lz4_frame", "zstd".
## Ignored unless mode is "custom".
#+ experimental_spill_compression = "uncompressed"
# --- End spill-to-disk ---
## The data storage options.
[storage]
## The working home directory.
data_home = "./greptimedb_data"
## Root directory for standalone SQL access to local files.
## Relative SQL paths are resolved below this directory. Absolute paths are accepted only when
## they are inside this directory. Defaults to `<data_home>/copy`.
## Distributed deployments always reject SQL access to local files.
## Upgrade note: COPY commands and existing external tables that reference paths outside this
## directory will fail. Move those files below the copy root, set this option to a dedicated
## directory containing them, or migrate the files to object storage before upgrading.
#+ copy_root = "./greptimedb_data/copy"
## The storage type used to store the data.
## - `File`: the data is stored in the local file system.
## - `S3`: the data is stored in the S3 object storage.
## - `Gcs`: the data is stored in the Google Cloud Storage.
## - `Azblob`: the data is stored in the Azure Blob Storage.
## - `Oss`: the data is stored in the Aliyun OSS.
## - `Hdfs`: the data is stored in the Hadoop Distributed File System.
type = "File"
## The S3 bucket name.
## **It's only used when the storage type is `S3`, `Oss` and `Gcs`**.
## @toml2docs:none-default
bucket = "greptimedb"
## The directory or object prefix under which data is stored.
## **It's only used when the storage type is `S3`, `Oss`, `Gcs`, `Azblob` and `Hdfs`**.
## @toml2docs:none-default
root = "greptimedb"
## The HDFS NameNode URI, for example, `hdfs://127.0.0.1:9000`.
## **It's only used when the storage type is `Hdfs`**.
## @toml2docs:none-default
name_node = "hdfs://127.0.0.1:9000"
## Additional options passed to the native HDFS client.
## **It's only used when the storage type is `Hdfs`**.
## @toml2docs:none-default
options = {}
## The access key id of the aws account.
## It's **highly recommended** to use AWS IAM roles instead of hardcoding the access key id and secret key.
## **It's only used when the storage type is `S3` and `Oss`**.
## @toml2docs:none-default
access_key_id = "test"
## The secret access key of the aws account.
## It's **highly recommended** to use AWS IAM roles instead of hardcoding the access key id and secret key.
## **It's only used when the storage type is `S3`**.
## @toml2docs:none-default
secret_access_key = "test"
## The secret access key of the aliyun account.
## **It's only used when the storage type is `Oss`**.
## @toml2docs:none-default
access_key_secret = "test"
## The account key of the azure account.
## **It's only used when the storage type is `Azblob`**.
## @toml2docs:none-default
account_name = "test"
## The account key of the azure account.
## **It's only used when the storage type is `Azblob`**.
## @toml2docs:none-default
account_key = "test"
## The scope of the google cloud storage.
## **It's only used when the storage type is `Gcs`**.
## @toml2docs:none-default
scope = "test"
## The credential path of the google cloud storage.
## **It's only used when the storage type is `Gcs`**.
## @toml2docs:none-default
credential_path = "test"
## The credential of the google cloud storage.
## **It's only used when the storage type is `Gcs`**.
## @toml2docs:none-default
credential = "base64-credential"
## The container of the azure account.
## **It's only used when the storage type is `Azblob`**.
## @toml2docs:none-default
container = "greptimedb"
## The sas token of the azure account.
## **It's only used when the storage type is `Azblob`**.
## @toml2docs:none-default
sas_token = ""
## The endpoint of the S3 service.
## **It's only used when the storage type is `S3`, `Oss`, `Gcs` and `Azblob`**.
## @toml2docs:none-default
endpoint = "https://s3.amazonaws.com"
## The region of the S3 service.
## **It's only used when the storage type is `S3`, `Oss`, `Gcs` and `Azblob`**.
## @toml2docs:none-default
region = "us-west-2"
## The http client options to the storage.
## **It's only used when the storage type is `S3`, `Oss`, `Gcs` and `Azblob`**.
[storage.http_client]
## The maximum idle connection per host allowed in the pool.
pool_max_idle_per_host = 1024
## The timeout for only the connect phase of a http client.
connect_timeout = "30s"
## The total request timeout, applied from when the request starts connecting until the response body has finished.
## Also considered a total deadline.
timeout = "30s"
## The timeout for idle sockets being kept-alive.
pool_idle_timeout = "90s"
## To skip the ssl verification
## **Security Notice**: Setting `skip_ssl_validation = true` disables certificate verification, making connections vulnerable to man-in-the-middle attacks. Only use this in development or trusted private networks.
skip_ssl_validation = false
# Custom storage options
# [[storage.providers]]
# name = "S3"
# type = "S3"
# bucket = "greptimedb"
# root = "data"
# access_key_id = "test"
# secret_access_key = "123456"
# endpoint = "https://s3.amazonaws.com"
# region = "us-west-2"
# [[storage.providers]]
# name = "Gcs"
# type = "Gcs"
# bucket = "greptimedb"
# root = "data"
# scope = "test"
# credential_path = "123456"
# credential = "base64-credential"
# endpoint = "https://storage.googleapis.com"
## The region engine options. You can configure multiple region engines.
## Each engine type (mito, file, metric) may appear only once; duplicates cause startup to fail.
[[region_engine]]
## The Mito engine options.
[region_engine.mito]
## Number of region workers. Defaults to half of the available CPU cores, rounded up.
## @toml2docs:none-default="Auto"
#+ num_workers = 8
## Request channel size of each worker.
worker_channel_size = 128
## Max batch size for a worker to handle requests.
worker_request_batch_size = 64
## Number of meta action updated to trigger a new checkpoint for the manifest.
manifest_checkpoint_distance = 10
## Whether to compress manifest and checkpoint file by gzip (default false).
compress_manifest = false
## Under development; do not enable. Whether to enable series indexes.
## Indexes are stored on the local filesystem under `{data_home}/series_index`.
#+ experimental_enable_series_index = false
## Approximate series and range index size limit in open regions, shared across workers.
## Workers periodically refresh usage and skip maintenance when full. In-flight reconciliation
## can exceed the limit. Closed-region files, temporary output, catalogs, and old snapshots
## retained by readers are not counted.
## Minimum: 1KiB. Takes effect on restart.
#+ experimental_series_index_max_size = "5GiB"
## Whether to build and query range indexes when series indexes are enabled.
## Obsolete range-index metadata and files are still cleaned up when disabled.
#+ experimental_enable_range_index = false
## Interval between series-index maintenance runs. Zero uses the default of 5 min.
#+ experimental_series_index_maintenance_interval = "5m"
## Requested minimum series-index bucket width (default: 5 days), rounded up to
## an exact multiple of each region's compaction time window.
#+ experimental_series_index_bucket_width = "5days"
## Max number of running background flush jobs (default: 1/2 of cpu cores).
## @toml2docs:none-default="Auto"
#+ max_background_flushes = 4
## Max number of running background compaction jobs (default: 1/4 of cpu cores).
## @toml2docs:none-default="Auto"
#+ max_background_compactions = 2
## Max number of running background purge jobs (default: number of cpu cores).
## @toml2docs:none-default="Auto"
#+ max_background_purges = 8
## Memory budget for compaction tasks.
## Supports absolute size (e.g., "2GiB", "512MB") or percentage of system memory (e.g., "50%").
## Setting it to 0 or "unlimited" disables the limit.
## @toml2docs:none-default="0"
#+ experimental_compaction_memory_limit = "0"
## Behavior when compaction cannot acquire memory from the budget.
## Options: "wait" (default, 10s), "wait(<duration>)", "fail"
## @toml2docs:none-default="wait"
#+ experimental_compaction_on_exhausted = "wait"
## Interval to auto flush a region if it has not flushed yet.
auto_flush_interval = "10m"
## Global write buffer size for all regions. If not set, it's default to 1/8 of OS memory with a max limitation of 1GB.
## @toml2docs:none-default="Auto"
#+ global_write_buffer_size = "1GB"
## Global write buffer size threshold to reject write requests. If not set, it's default to 2 times of `global_write_buffer_size`.
## @toml2docs:none-default="Auto"
#+ global_write_buffer_reject_size = "2GB"
## Default write buffer size for each region. Regions stall at this size and reject writes at twice this size. Setting it to 0 disables both limits unless the table specifies `write_buffer_size`.
#+ default_region_write_buffer_size = "0"
## Cache size for SST metadata. Setting it to 0 to disable the cache.
## If not set, it's default to 1/8 of OS memory with a max limitation of 512MB.
## @toml2docs:none-default="Auto"
#+ sst_meta_cache_size = "512MB"
## Cache size for vectors and arrow arrays. Setting it to 0 to disable the cache.
## If not set, it's default to 1/16 of OS memory with a max limitation of 512MB.
## @toml2docs:none-default="Auto"
#+ vector_cache_size = "512MB"
## Cache size for pages of SST row groups. Setting it to 0 to disable the cache.
## If not set, it's default to 1/8 of OS memory.
## @toml2docs:none-default="Auto"
#+ page_cache_size = "512MB"
## Cache size for time series selector (e.g. `last_value()`). Setting it to 0 to disable the cache.
## If not set, it's default to 1/16 of OS memory with a max limitation of 512MB.
## @toml2docs:none-default="Auto"
#+ selector_result_cache_size = "512MB"
## Cache size for flat range scan results. Setting it to 0 to disable the cache.
## If not set, it's default to 1/16 of OS memory with a max limitation of 512MB.
## @toml2docs:none-default="Auto"
#+ range_result_cache_size = "512MB"
## Cache size for prefilter results. Setting it to 0 to disable the cache.
## If not set, it's default to 1/32 of OS memory with a max limitation of 128MB.
## @toml2docs:none-default="Auto"
#+ prefilter_result_cache_size = "128MB"
## Whether to enable the write cache, it's enabled by default when using object storage. It is recommended to enable it when using object storage for better performance.
enable_write_cache = false
## File system path for write cache, defaults to `{data_home}`.
write_cache_path = ""
## Capacity for write cache. If your disk space is sufficient, it is recommended to set it larger.
write_cache_size = "5GiB"
## TTL for write cache.
## @toml2docs:none-default
write_cache_ttl = "8h"
## Preload index (puffin) files into cache on region open (default: true).
## When enabled, index files are loaded into the write cache during region initialization,
## which can improve query performance at the cost of longer startup times.
preload_index_cache = true
## Percentage of write cache capacity allocated for index (puffin) files (default: 20).
## The remaining capacity is used for data (parquet) files.
## Must be between 0 and 100 (exclusive). For example, with a 5GiB write cache and 20% allocation,
## 1GiB is reserved for index files and 4GiB for data files.
index_cache_percent = 20
## Enable refilling cache on read operations (default: true).
## When disabled, cache refilling on read won't happen.
enable_refill_cache_on_read = true
## Capacity for manifest cache (default: 256MB).
manifest_cache_size = "256MB"
## Buffer size for SST writing.
sst_write_buffer_size = "8MB"
## Maximum number of SST files to scan concurrently.
max_concurrent_scan_files = 384
## Whether to allow stale WAL entries read during replay.
allow_stale_entries = false
## Memory limit for table scans across all queries.
## Supports absolute size (e.g., "2GB") or percentage of system memory (e.g., "20%").
## Setting it to 0 or "unlimited" disables the limit.
scan_memory_limit = "unlimited"
## Controls what happens when a scan cannot get memory immediately.
## "fail" (default) fails fast and is the recommended option for most users.
## "wait" / "wait(<duration>)" waits for memory to become available. This is mainly
## for advanced tuning in bursty workloads where temporary contention is common and
## higher latency is acceptable.
## "wait" means "wait(10s)", not unlimited waiting.
scan_memory_on_exhausted = "fail"
## Minimum time interval between two compactions.
## To align with the old behavior, the default value is 0 (no restrictions).
min_compaction_interval = "0m"
## Whether to allow to schedule a compaction after a successful region edit.
##
## Setting this to "true" is a necessary but not sufficient condition for scheduling compaction after a region edit.
## Other constraints, such as "min_compaction_interval", may still prevent compaction from being scheduled.
## Setting this to "false", however, guarantees that compaction will not be scheduled after a region edit.
schedule_compaction_after_edit = true
## Whether to enable flat format as the default SST format.
default_flat_format = true
## Whether to enable the experimental two-phase mode for eligible metric series scans.
experimental_series_scan_v2 = false
## The options for index in Mito engine.
[region_engine.mito.index]
## Auxiliary directory path for the index in filesystem, used to store intermediate files for
## creating the index and staging files for searching the index, defaults to `{data_home}/index_intermediate`.
## The default name for this directory is `index_intermediate` for backward compatibility.
##
## This path contains two subdirectories:
## - `__intm`: for storing intermediate files used during creating index.
## - `staging`: for storing staging files used during searching index.
aux_path = ""
## The max capacity of the staging directory.
staging_size = "2GB"
## The TTL of the staging directory.
## Defaults to 7 days.
## Setting it to "0s" to disable TTL.
staging_ttl = "7d"
## Cache size for inverted index metadata.
metadata_cache_size = "64MiB"
## Cache size for inverted index content.
content_cache_size = "128MiB"
## Page size for inverted index content cache.
content_cache_page_size = "64KiB"
## Cache size for index result.
result_cache_size = "128MiB"
## The options for inverted index in Mito engine.
[region_engine.mito.inverted_index]
## Whether to create the index on flush.
## - `auto`: automatically (default)
## - `disable`: never
create_on_flush = "auto"
## Whether to create the index on compaction.
## - `auto`: automatically (default)
## - `disable`: never
create_on_compaction = "auto"
## Whether to apply the index on query
## - `auto`: automatically (default)
## - `disable`: never
apply_on_query = "auto"
## Memory threshold for performing an external sort during index creation.
## - `auto`: automatically determine the threshold based on the system memory size (default)
## - `unlimited`: no memory limit
## - `[size]` e.g. `64MB`: fixed memory threshold
mem_threshold_on_create = "auto"
## Deprecated, use `region_engine.mito.index.aux_path` instead.
intermediate_path = ""
## The options for full-text index in Mito engine.
[region_engine.mito.fulltext_index]
## Whether to create the index on flush.
## - `auto`: automatically (default)
## - `disable`: never
create_on_flush = "auto"
## Whether to create the index on compaction.
## - `auto`: automatically (default)
## - `disable`: never
create_on_compaction = "auto"
## Whether to apply the index on query
## - `auto`: automatically (default)
## - `disable`: never
apply_on_query = "auto"
## Memory threshold for index creation.
## - `auto`: automatically determine the threshold based on the system memory size (default)
## - `unlimited`: no memory limit
## - `[size]` e.g. `64MB`: fixed memory threshold
mem_threshold_on_create = "auto"
## The options for bloom filter in Mito engine.
[region_engine.mito.bloom_filter_index]
## Whether to create the bloom filter on flush.
## - `auto`: automatically (default)
## - `disable`: never
create_on_flush = "auto"
## Whether to create the bloom filter on compaction.
## - `auto`: automatically (default)
## - `disable`: never
create_on_compaction = "auto"
## Whether to apply the bloom filter on query
## - `auto`: automatically (default)
## - `disable`: never
apply_on_query = "auto"
## Memory threshold for bloom filter creation.
## - `auto`: automatically determine the threshold based on the system memory size (default)
## - `unlimited`: no memory limit
## - `[size]` e.g. `64MB`: fixed memory threshold
mem_threshold_on_create = "auto"
[[region_engine]]
## Enable the file engine.
[region_engine.file]
[[region_engine]]
## Metric engine options.
[region_engine.metric]
## The logging options.
[logging]
## The directory to store the log files. If set to empty, logs will not be written to files.
dir = "./greptimedb_data/logs"
## The log level. Can be `info`/`debug`/`warn`/`error`.
## @toml2docs:none-default
level = "info"
## Enable OTLP tracing.
enable_otlp_tracing = false
## The OTLP tracing endpoint.
otlp_endpoint = "http://localhost:4318/v1/traces"
## Whether to append logs to stdout.
append_stdout = true
## Whether to write logs to files in `dir`.
enable_file_logging = true
## The log format. Can be `text`/`json`.
log_format = "text"
## The maximum amount of log files.
max_log_files = 720
## The maximum total size of managed log files in `dir`.
## Old closed log files are removed before writing when necessary. Active files may exceed it. Set to `0B` to disable.
max_log_dir_size = "0B"
## Whether to enable per-region metrics.
## Default to false.
enable_per_region_metrics = false
## The OTLP tracing export protocol. Can be `grpc`/`http`.
otlp_export_protocol = "http"
## Additional OTLP headers, only valid when using OTLP http
[logging.otlp_headers]
## @toml2docs:none-default
#Authorization = "Bearer my-token"
## @toml2docs:none-default
#Database = "My database"
## The percentage of tracing will be sampled and exported.
## Valid range `[0, 1]`, 1 means all traces are sampled, 0 means all traces are not sampled, the default value is 1.
## ratio > 1 are treated as 1. Fractions < 0 are treated as 0
[logging.tracing_sample_ratio]
default_ratio = 1.0
## The slow query log options.
[slow_query]
## Whether to enable slow query log.
enable = true
## The record type of slow queries. It can be `system_table` or `log`.
## If `system_table` is selected, the slow queries will be recorded in a system table `greptime_private.slow_queries`.
## If `log` is selected, the slow queries will be logged in a log file `greptimedb-slow-queries.*`.
record_type = "system_table"
## The threshold of slow query. It can be human readable time string, for example: `10s`, `100ms`, `1s`.
threshold = "30s"
## The sampling ratio of slow query log. The value should be in the range of (0, 1]. For example, `0.1` means 10% of the slow queries will be logged and `1.0` means all slow queries will be logged.
sample_ratio = 1.0
## The TTL of the `slow_queries` system table. Default is `90d` when `record_type` is `system_table`.
ttl = "90d"
## The tracing options. Only effect when compiled with `tokio-console` feature.
#+ [tracing]
## The tokio console address.
## @toml2docs:none-default
#+ tokio_console_addr = "127.0.0.1"
## The pipeline options.
[pipeline]
## Time to live of the local pipeline cache. Default is `10s`.
cache_ttl = "10s"
## Allow VRL to access the server environment, filesystem, hostname, local timezone,
## network, and logs. Default is false. Enable only for trusted pipeline authors.
## Applies to pipeline creation, execution, and dry-runs. Requires a restart.
allow_vrl_host_access = false
## Configuration options for the event recorder.
[event_recorder]
## TTL for the events table that will be used to store the events. Default is `90d`.
ttl = "90d"
## Event types to record. Current available event types: `create_database`,
## `alter_database`, `drop_database`, `create_flow`, `drop_flow`,
## `create_table`, `create_logical_tables`, `alter_table`, `alter_logical_tables`,
## `drop_table`, `undrop_table`, `purge_dropped_table`, `truncate_table`,
## `create_view`, `drop_view`, `admin_function`, `reconcile_catalog`,
## `reconcile_database`, `reconcile_table`, `reconcile_logical_tables`.
## When omitted, all current and future event types are recorded.
## Set to an empty array to disable event recording.
## @toml2docs:none-default="All"
#+ event_types = ["create_database"]
## The memory options.
[memory]
## Whether to enable heap profiling activation during startup.
## When enabled, heap profiling will be activated if the `MALLOC_CONF` environment variable
## is set to "prof:true,prof_active:false". The official image adds this env variable.
## Default is true.
enable_heap_profiling = true