greptimedb/docs/scanbench.md

Scanbench Usage

scanbench benchmarks region scans directly from storage through:

greptime datanode scanbench ...

Build

cargo build -p cmd --bin greptime --features dev-tools

Command

./target/debug/greptime datanode scanbench \
  --config <CONFIG_TOML> \
  --region-id <REGION_ID> \
  --table-dir <TABLE_DIR> \
  [--scanner <seq|unordered|series>] \
  [--scan-config <SCAN_CONFIG_JSON>] \
  [--scan-configs <SCAN_CONFIGS_JSON>] \
  [--parallelism <N>] \
  [--iterations <N>] \
  [--path-type <bare|data|metadata>] \
  [--enable-wal] \
  [--pprof-file <FLAMEGRAPH_SVG>] \
  [--result-file <RESULT_JSON>] \
  [--pprof-after-warmup] \
  [--verbose]

Required Arguments

• --config: Datanode/standalone TOML config.
• --region-id: Region ID in one of:
  • <u64> (example: 4398046511104)
  • <table_id>:<region_number> (example: 1024:0)
• --table-dir: Table directory used in open request (example: greptime/public/1024).

Optional Arguments

• --scanner: Scan strategy. Default: seq.
  • seq: default scan
  • unordered: time-windowed distribution
  • series: per-series distribution
• --scan-config: JSON file to tune scan request.
• --scan-configs: JSON array of named scan requests. Each entry is executed exactly once in file order. This conflicts with --scan-config; the array length replaces --iterations, which must remain at its default value of 1.
• --parallelism: Simulated scan parallelism. Default: 1.
• --iterations: Benchmark iterations. Default: 1.
• --path-type: Region path type (bare, data, metadata). Default: bare.
• --enable-wal: Enable WAL replay when opening the region. Default: disabled. When enabled, scanbench uses the log store configured in the [wal] section of the config TOML (raft-engine or Kafka). When disabled or when no WAL is configured, a NoopLogStore is used.
• --pprof-file: Output flamegraph path (Unix only).
• --result-file: Write structured benchmark and scanner analyze results as JSON. Existing files are overwritten after all scans complete successfully. Supplying this option collects verbose scanner metrics even without --verbose.
• --pprof-after-warmup: Start profiling after the first iteration, using it as a warmup. Requires --pprof-file. Default: disabled.
• --verbose / -v: Print scanner metrics plus per-partition row counts, first-batch latency, total elapsed time, and partition skew.

Scan Config JSON

{
  "projection": [0, 1, 2],
  "projection_names": ["host", "cpu"],
  "filters": ["host = 'web-1'", "cpu > 80"],
  "series_row_selector": "last_row"
}

Notes:

• All fields are optional.
• Use either projection (indexes) or projection_names (column names), not both.
• projection_names uses exact (case-sensitive) column name matching.
• filters is a list of SQL expressions (not full SQL statements), e.g. "host = 'web-1'".
• series_row_selector currently supports only "last_row".

Multiple Scan Configs

Use --scan-configs to run a sequence of different scan requests:

[
  {
    "name": "cold",
    "projection_names": ["hostname", "usage_user"],
    "filters": ["hostname = 'host_1'"]
  },
  {
    "name": "hot-001",
    "projection_names": ["hostname", "usage_user"],
    "filters": ["hostname = 'host_2'"]
  }
]

The array must contain at least one config. name is optional; omitted names become query-001, query-002, and so on. Names must be non-empty and unique. Scanbench validates every config before starting the benchmark, executes the array once in order, and prints both an overall mean and a per-query summary.

With --pprof-after-warmup, the first entry is the warmup query and profiling starts before the second entry. The warmup query remains part of the reported overall statistics.

Result JSON

--result-file writes a versioned JSON document with benchmark settings, ordered run results, and overall and per-query summaries. Each run contains its normalized config, row and batch counts, setup/scan/total timing, memory sizes, per-partition statistics, and verbose scanner explain output. Normalized projections use resolved column indexes.

Timing fields end in _ns and use nanoseconds. Size fields end in _bytes. If config validation or any scan fails, scanbench returns the error without writing the result file.

Examples

Default sequential scan:

./target/debug/greptime datanode scanbench \
  --config /path/to/config.toml \
  --region-id 1024:0 \
  --table-dir greptime/public/1024

Unordered scan with parallelism:

./target/debug/greptime datanode scanbench \
  --config /path/to/config.toml \
  --region-id 1024:0 \
  --table-dir greptime/public/1024 \
  --scanner unordered \
  --parallelism 8 \
  --iterations 5

Run 10 different queries once each by placing 10 entries in the JSON array:

./target/debug/greptime datanode scanbench \
  --config /path/to/config.toml \
  --region-id 1024:0 \
  --table-dir greptime/public/1024 \
  --scanner seq \
  --parallelism 8 \
  --scan-configs /path/to/scan-configs.json \
  --result-file /path/to/scanbench-results.json \
  --verbose

Series scan with scan config and flamegraph:

./target/debug/greptime datanode scanbench \
  --config /path/to/config.toml \
  --region-id 1024:0 \
  --table-dir greptime/public/1024 \
  --scanner series \
  --scan-config /path/to/scan-config.json \
  --pprof-file /tmp/scanbench.svg

Scan with WAL replay enabled (uses [wal] config from TOML):

./target/debug/greptime datanode scanbench \
  --config /path/to/config.toml \
  --region-id 1024:0 \
  --table-dir greptime/public/1024 \
  --enable-wal