diff --git a/config/config.md b/config/config.md index b4fb40c477..a717b0d613 100644 --- a/config/config.md +++ b/config/config.md @@ -514,7 +514,7 @@ | `wal.timeout` | String | `3s` | The timeout for kafka client.
**It's only used when the provider is `kafka`**. | | `wal.max_batch_bytes` | String | `1MB` | The max size of a single producer batch.
Warning: Kafka has a default limit of 1MB per message in a topic.
**It's only used when the provider is `kafka`**. | | `wal.consumer_wait_timeout` | String | `100ms` | The consumer wait timeout.
**It's only used when the provider is `kafka`**. | -| `wal.create_index` | Bool | `true` | Whether to enable WAL index creation.
**It's only used when the provider is `kafka`**. | +| `wal.create_index` | Bool | `false` | Whether to enable WAL index creation.
**It's only used when the provider is `kafka`**. | | `wal.dump_index_interval` | String | `60s` | The interval for dumping WAL indexes.
**It's only used when the provider is `kafka`**. | | `wal.overwrite_entry_start_id` | Bool | `false` | Ignore missing entries during read WAL.
**It's only used when the provider is `kafka`**.

This option ensures that when Kafka messages are deleted, the system
can still successfully replay memtable data without throwing an
out-of-range error.
However, enabling this option might lead to unexpected data loss,
as the system will skip over missing entries instead of treating
them as critical errors. | | `query` | -- | -- | The query engine options. | diff --git a/config/datanode.example.toml b/config/datanode.example.toml index 678af44376..7a5a486bbe 100644 --- a/config/datanode.example.toml +++ b/config/datanode.example.toml @@ -185,7 +185,7 @@ consumer_wait_timeout = "100ms" ## Whether to enable WAL index creation. ## **It's only used when the provider is `kafka`**. -create_index = true +create_index = false ## The interval for dumping WAL indexes. ## **It's only used when the provider is `kafka`**. diff --git a/src/common/wal/src/config/kafka/datanode.rs b/src/common/wal/src/config/kafka/datanode.rs index 374a7229d4..4003b52e4b 100644 --- a/src/common/wal/src/config/kafka/datanode.rs +++ b/src/common/wal/src/config/kafka/datanode.rs @@ -60,7 +60,7 @@ impl Default for DatanodeKafkaConfig { consumer_wait_timeout: Duration::from_millis(100), kafka_topic: KafkaTopicConfig::default(), auto_create_topics: true, - create_index: true, + create_index: false, dump_index_interval: Duration::from_secs(60), topic_latest_offset_fetch_interval: DEFAULT_TOPIC_LATEST_OFFSET_FETCH_INTERVAL, overwrite_entry_start_id: false,