* fix(mito2): cancel cache construction for incomplete scans
CacheBatchBuffer spawned the background concat task and dropped its join
handle. A scan that was cancelled or failed therefore left the task alive:
it kept compacting already queued batches, and while waiting for a range
result memory permit it held them, even though without a finish command
the result can never be put into the cache.
Keep the handle and abort it when the buffer is dropped. The handle is
cleared once the task owns the finish command, so a completed scan still
populates the cache after its stream is dropped. Abort does not preempt a
concat that is already running; it takes effect the next time the task is
polled.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(mito2): wait for the permit park before cancelling the buffer
An empty buffered_batches only proves the batches were enqueued, so the
cancellation test could abort a concat task that had never been polled.
Count acquisitions that find too few permits, a test-only signal, and drop
the buffer once the task has reached that wait. Nothing awaits between the
check and the parking, so an observed increment means the caller is about
to wait for permits the test holds for the rest of the case.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: add config flag to control range index reads
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: disable range index builds when configured and default to off
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
CompactDispatcher acquired a borrowed permit inside the async wrapper and
dropped it as soon as the blocking task was submitted, so the semaphore
never bounded the compactions that actually ran. Acquire an owned permit
and move it into the blocking closure so it covers the queue wait and the
merge itself.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(prometheus): honor label matchers in __name__ values query
`/api/v1/label/__name__/values?match[]={pod="abc"}` dropped every matcher
other than `__name__` and returned all metrics in the schema. No error,
just the wrong list. Grafana's metrics browser sends this request, so
picking a label value there did nothing.
Selectors that only constrain `__name__` keep answering from table
metadata. A selector constraining an ordinary label now goes to the data:
scan each metric engine physical table for distinct `__table_id` in the
time range, map the ids back to metric names, then apply the selector's
own `__name__` matchers.
Only metric engine tables are covered; other engines share no column space
to scan.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(prometheus): batch-resolve metric names by table id
Building a full table-id-to-name map meant walking every table in the
schema and holding all of them in memory, just to name the handful the
scan returned. Use `tables_by_ids` instead — one batch KV read over the
ids the scan actually produced.
The catalog walk stays, but only to find the physical tables to scan.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): read an absent label as the empty string
A matcher on a label the series does not carry only worked when the table
had no column for it at all. Where the column exists but is NULL on that
row -- the norm for logical metrics sharing a metric engine physical
table, which holds the union of their label columns -- three-valued logic
dropped the row, so `host!="host1"` and `host=""` missed every metric
without a host label.
Coalesce nullable string label columns to "" for matchers that accept the
empty string, rather than only for the OTLP temporality marker. Equality
matchers are untouched; they cannot match NULL either way.
This is the Prometheus compatibility fix#8970 deliberately kept out of
its own scope. The cost is visible in the regex sqlness plan: the
predicate becomes a CASE, so the scan loses its LastRow selector and
grows a FilterExec. Only negative and empty-accepting matchers pay it.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): don't panic on pre-epoch label value bounds
`rewrite_label_values_query` unwrapped `duration_since(UNIX_EPOCH)`, which
returns an error for an instant before the epoch. `start=1969-12-31T23:59:59Z`
parses as valid RFC3339, so the request panicked instead of answering.
Recover the sign from the error branch, and report a value beyond i64
milliseconds as an error rather than wrapping the cast.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(prometheus): drop applicable_matchers, share the distinct scan
With the planner reading an absent label as empty, the frontend no longer
needs to pre-filter matchers per physical table. Removing that exposed a
second problem: a physical table that never took a column from a logical
table exposes no `__table_id`, and projecting it failed the whole request.
Skip those tables; the only thing that can miss is a metric with no labels.
Also pulls out the plan-build-execute-collect sequence the two label value
scans had in common.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Agents routinely check "This PR requires documentation updates" for
changes users never see. Say what the item means: the docs site repo,
not rustdoc or in-repo comments.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
crate-ci/typos renamed its default branch from master to main on
2026-09-18 (0a3d75e) and the master branch is gone, so every workflow
run since then fails at job setup with:
Unable to resolve action `crate-ci/typos@master`, unable to find version `master`
Pin to the latest release tag instead of tracking a branch. This matches
how the other actions in these two workflows are referenced and keeps the
check reproducible.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Incremental batching flows already merge sink state for scalar
aggregates and the HLL/UDDSketch/stddev state families. AVG now exposes
a mergeable Binary state (avg_state / avg_merge with
__avg_state_delta_merge), so admit those aggregates too instead of
forcing a full snapshot for flows whose only state column is an
average.
merge_op_for_aggregate_expr takes the aggregate input schema so the
avg_merge arm can require a Binary state argument; the state form is
the aggregate result persisted by the sink, so no extra coercion is
needed. Other input types keep being rejected.
Also cover avg_state, avg_merge and duplicate AVG projections in the
incremental plan analysis tests, extend the mixed state-family rewrite
test with an average column, and extend the standalone partitioned
state-merge SQLness case with avg_state/avg_merge compared against a
direct avg over the source.
Signed-off-by: discord9 <discord9@outlook.com>
Co-authored-by: discord9 <discord9@outlook.com>
* test(flow): wait for async source-table mirrors before FLUSH_FLOW
Streaming flows execute inline in the flownode insert handler since
#8976, and source-table inserts are mirrored from the frontend as
detached tasks. Tests that INSERT then ADMIN FLUSH_FLOW then SELECT
intermittently miss rows on slow CI runners because the flush no longer
implies the mirror reached the flownode (flow_no_aggr lost row 'l',
flow_advance_ttl lost row '23' after the post-restart reinsert).
Add SQLNESS SLEEP 3s between those mirror inserts and the flush, the
established pattern in the flow suite, so the assertions are stable.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(flow): also guard first-section flushes in flow_advance_ttl
Local reproduction without sleeps failed in a window the previous
commit did not cover: the first INSERT (20,20,22) of each section is
followed immediately by ADMIN FLUSH_FLOW, and the distributed run
observed an empty sink on that SELECT (~1 in 8 iterations). Add the
same SLEEP 3s guard before those two flushes.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(promql): propagate matching-label filters between binary operands
A one-to-one arithmetic binary expression inner-joins its operands on the
matching labels, so every row that survives the join already satisfies the
other operand's equality matchers on those labels. Copy those matchers to the
other operand so both scans drop non-joining series before execution instead of
feeding them to the join.
Both operands are planned before the rewrite: a selector matcher can also
constrain a value field, and only the planned contexts tell tags and fields
apart. A matcher is copied only when its name is a tag column on both sides, its
value is non-empty, and it is one of the matching labels. Operands are limited
to vector selectors, parentheses, and label-preserving range functions applied
directly to a matrix selector; `ignoring(...)`, group modifiers, fill values,
set and comparison operators, regex matchers and `or` matcher groups are left
alone.
Selector matchers are now deduplicated in place instead of through a `HashSet`,
so the generated scan filter keeps a stable order once a selector carries more
than one matcher.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* perf(promql): propagate ignoring, non-equality and aggregated matchers
Widens the matcher propagation added in the previous commit to the shapes it
was leaving on the table.
The join compares matching labels with plain column equality
(`normalized_match_key_expr` and its coalescing are confined to `or`), so any
predicate on a matching label is already enforced on both sides for every
surviving pair: it originates on one operand, and the join carries it to the
pairs it forms. Copying it to the other operand can only drop rows that had no
surviving partner. That argument does not depend on the matcher kind, so regular
expressions, negations and empty values now propagate too. `=~".*"` stays out:
it lowers to no filter at all, so copying it would only force a re-plan.
`ignoring(...)` is no longer rejected. A label is a join key exactly when it is a
tag on both sides and not named in `ignoring`, which is what
`binary_join_key_columns` computes and what the caller can now answer from the
two planned contexts.
Operands may now be aggregations that partition by their grouping labels, which
is the shape most real queries use. `agg_modifier_to_col` rewrites
`ctx.tag_columns` to the grouping labels, so a label found in an aggregated
operand's tags is a group key, and filtering the aggregate's input by it drops
exactly the corresponding output groups. `topk`, `bottomk` and `limitk` are
excluded because they select across a group and carry input labels through --
`prom_topk_bottomk_to_plan` leaves `ctx.tag_columns` alone, so the tag check
cannot catch them. `count_values` is excluded because it adds an output label
that does not exist in its input.
The rollup whitelist now covers every label-preserving range function the
planner implements, and finds the matrix argument by position so multi-argument
rollups such as `quantile_over_time` and `predict_linear` qualify.
`absent_over_time` stays out: it synthesizes a series from the matchers when its
input has none.
Re-running the sqlness case against an unmodified planner produces a
byte-identical `.result` for all 27 queries.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): pin that excluded matching labels stay on their own operand
`ignoring(device)` and `on(host)` both leave `device` out of the join keys, so a
`device` matcher must not reach the other operand. Neither case was covered end
to end: a regression there silently drops the left operand's `eth1` series
instead of returning them, which the two added queries now catch.
Also corrects the comment at the rewrite site, which still described re-planning
as touching a leaf selector. Aggregated operands are re-planned as a whole; what
holds is that the rewrite only ever adds matchers to a selector, so the enclosing
operand's table reference, time index and field columns are unchanged.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(promql): log why matching-filter propagation was skipped
Splits the guard chain into `try_propagate`, whose `Err` names the reason the
rewrite does not apply, and logs it once in `propagate`. Debugging a query that
did not get the filter no longer means stepping through the guards.
The reasons also replace the comments that used to explain the same conditions,
and the remaining comments lose the parts that restated the code or repeated
each other.
Two of them were wrong rather than verbose. Saying `topk`'s input "must not be
filtered" reads as a claim about PromQL: `topk(1, m{host="x"})` is perfectly
legal, and what matters is that filtering before `topk` changes the candidate
set it ranks. Saying the subset-matching case "keeps its many-to-many result"
read as an endorsement of behaviour Prometheus rejects outright; it now records
that the query is a cross product here and points at #9209.
The propagation cannot change whether such a check would fire: series in one
match group agree on every matching label, so a matcher over one of those labels
keeps all of them or drops all of them.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): keep copied matchers out of the operand's selector metadata
Re-planning an operand with a copied matcher also rebuilt its
`PromPlannerContext::selector_matcher`, and that context is what an enclosing
expression reads. `create_absent_plan` turns the equality matchers found there
into the labels `absent()` reports, so
absent(counter_metric{host="missing"} / on(host, device) gauge_metric)
gained a `host="missing"` label that `main` does not produce. The inner
expression was empty either way; only the reported label set changed.
The copied matcher belongs to the scan, not to the operand's identity, so both
re-planned contexts now keep the matchers their operand was written with.
`selector_matcher` has one other reader, `create_table_scan_plan`, which
consumes it while the operand is being planned and is unaffected.
Reported by @discord9.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(log-store): add the object store WAL batch, catalog and I/O
Add the three modules between the object format and the store of the
object store WAL:
- batch: the open batch that accumulates admitted entries into the next
object and assigns entry ids at admission. Ids are object-sequence-major,
`(object_seq << 20) | position`, with positions starting at one per
region per object, so a batch that is rolled back and admitted again
under the same sequence hands out the same ids. The time the first entry
was admitted is kept for the store's age-based sealing; an empty
admission does not start it.
- catalog: the in-memory index over object footers, by sequence and per
region. Insertion is atomic and rejects an empty footer, duplicate
region segments, invalid entry ranges, an already indexed sequence and
entry ranges that are not strictly increasing across objects. The next
object sequence continues after the largest indexed one and is raised
above the largest entry id of any region, and is rejected once it no
longer fits an entry id.
- io: object store access under `<prefix>/objects/`: a conditional create
whose retry with identical content is a no-op and whose conflicting
content is rejected, whole and range reads, and a listing of well-formed
keys in sequence order. A prefix with a `.` or `..` component is
rejected. When the store reports that the object exists but the read
that compares its content fails, the read failure is returned with its
retry hint instead of the create failure.
The modules have no callers until the store lands, so they are declared
with `#[allow(dead_code)]`. Only `entry_id` is exported.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(log-store): drop comments that restate the batch test assertions
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* refactor(log-store): use pub(crate) in the object store WAL batch, catalog and I/O
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(log-store): keep entry_id crate-private and narrow the create collision check
Re-export `entry_id` as `pub(crate)`: nothing outside `log-store` uses it
yet, and exporting it would freeze the raw `(object_seq, position)`
encoding before the store owns id allocation.
Treat only `ConditionNotMatch` as the sign that a conditional create
collided with an existing object. A store may report `AlreadyExists` for
an unrelated path, for example when a parent of the object is a file, and
that failure must come back as the write failure rather than as the error
of the read that would compare content.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* refactor(log-store): rename the catalog's out_of_order helper to out_of_order_reason
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* feat(function): add mergeable binary average states
Signed-off-by: discord9 <discord9@163.com>
* feat(function): expose avg_calc as an OSS scalar
Signed-off-by: discord9 <discord9@163.com>
* fix(function): borrow invalid AVG scalar argument type
Signed-off-by: discord9 <discord9@163.com>
* test(function): verify public AVG state SQL finalization
Signed-off-by: discord9 <discord9@163.com>
* fix(function): return canonical AVG1 state for empty window frames
DataFusion's plain-aggregate window executor bypasses the accumulator
and calls default_value() directly when a window frame contains no
rows. create_udaf's SimpleAggregateUDF derives default_value from the
return type, which yields SQL NULL for Binary output instead of the
canonical AVG1 empty state, so empty frames and frames over only NULL
inputs became observable differently (IS NULL, direct state saves,
state comparisons).
Register avg_state/avg_merge through a small AggregateUDFImpl (AvgUdaf)
that keeps the existing accumulator and declares the canonical AVG1
empty state as default_value, restoring the documented contract. Add a
unit test asserting default_value equals the empty accumulator's
evaluate() and an sqlness case covering the empty-frame window.
Signed-off-by: discord9 <discord9@outlook.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@outlook.com>
Co-authored-by: discord9 <discord9@outlook.com>
* perf: add flight coalesce regression case with high-cardinality aggregations
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf: add flight coalesce aggregations bench case
Add a direct_readable_sst case exercising grouped aggregation over
coalesced batches: 16 hosts x 4096 instances, 32 SSTs of 32768 rows,
timestamp-major series layout, three SQL queries (aggregation, topk,
count_by_host) each with a 10% max candidate latency regression
threshold.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* chore: bump memcomparable to d8fb3558 for bytes I/O optimizations
The new revision (v0y4g3r/memcomparable main, PR #1) includes:
- Deserializer::read_bytes_into for reusable-buffer decoding
- read_bytes 32B pre-allocation (no more 0→8→16→32 realloc chain)
- bulk put_slice in both normal and reverse modes
- Error::Eof instead of panic on truncated input
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* perf(mito2): extract sparse tag values lazily without full PK decode
Replace the full primary-key decode in two hot consumers of
SortField::deserialize with column-lazy extraction over
SparsePrimaryKeyView, eliminating the per-label owned values
(Vec<Value>, one heap allocation per string) for every distinct key:
- flat_format: DecodedPrimaryKeys keeps sparse keys encoded and
extracts each tag column directly from the raw dictionary bytes into
a reusable buffer. FlatConvertFormat (flush/compaction write path)
builds all projected tag columns in one pass per key, sharing offset
discovery between columns; the read path (file_range filters) keeps
per-column lazy extraction.
- series_index writer: builds rows from the reserved prefix and only
the indexed tags via a shared offsets cache and buffer, instead of
decoding all labels and copying out a few strings.
- mito-codec: add SparsePrimaryKeyView::reserved_value for typed
table_id/tsid extraction.
Persisted formats are unchanged; values pushed to column builders are
identical (NULL vs empty string, reserved column types, duplicate
dictionary entries and consecutive-run order are preserved).
Benchmark (bench_pk_tag_column, 4096 rows, 40 labels per key, Criterion
30 samples): old = full decode + per-column builds, new = lazy
extraction (+ one-pass for multi-column projection).
| Workload | Before | After | Change |
| --- | ---: | ---: | ---: |
| 1 of 40 tags, 4096 distinct keys | 6.95 ms | 803 us | -88% |
| 1 of 40 tags, 128 distinct keys | 206.8 us | 29.2 us | -86% |
| 10 of 40 tags, 128 distinct keys | 236.2 us | 75.5 us | -68% |
| 40 of 40 tags, 128 distinct keys | 314 us | 215.2 us | -31% |
| 1 of 10 tags, 4096 distinct keys | 1.75 ms | 207.6 us | -88% |
Also adds a decode_sparse benchmark group to mito-codec's
bench_sparse_encoding covering 10/40 labels x 10/24/64-byte values.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): share sparse tag scans across filters
Batch missing tag columns used by simple filters and partition pruning, while retaining single-column extraction and skip-mode semantics.
Replace the optional reserved-byte accessor with typed table_id/tsid reads from the validated prefix. Add full-decode equivalence tests for sliced dictionaries, NULL/empty labels and cross-key scratch reuse, plus series-index roundtrips across batches.
Benchmark real precise filtering with 4096 rows: 10 tag predicates drop from 307.85 to 162.49 us; 40 predicates drop from 894.03 to 550.55 us. All 1459 mito2/mito-codec tests pass.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test(mito2): benchmark sparse tag predicate count scaling
Add a dedicated precise-filter benchmark group for 1, 2, 4, 8, 16, and 32 predicates with both 1 and 32 rows per primary key. Keep 4096 rows, 40 labels, and full selectivity constant across cases, and validate the workload outside the timed section.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat(log-store): add the object store WAL object format
Add the byte format of a single object store WAL object: a header with
the GTWALOBJ magic, format version 1, the object sequence and the writer
instance id; one segment per region ordered by region id with entries
ordered by entry id; a footer that records each segment's region id,
entry id range, entry count, byte range and CRC32; and a fixed trailer
with the GTWALTRL magic, the footer location, the footer CRC32 and the
whole-object CRC32.
The module encodes objects deterministically and decodes the header,
trailer, footer and segments separately, with structural checks on
footer ranges and segment tiling. It has no callers yet; the store that
writes and reads objects follows in later changes.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(log-store): pin the object store WAL format with a byte fixture
Add a fixed version 1 object with two regions and five entries as a hex
literal. The test decodes it and checks the exact header, trailer,
footer entries and records, and checks that encoding the same records,
in either input order, reproduces the fixture byte for byte.
Round-trip tests alone pass when a refactor changes field order,
endianness or checksum coverage in both the encoder and the decoder.
The fixture bytes were derived from the documented layout rather than
from the encoder, so such a change now fails.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(log-store): reject an empty footer when decoding a WAL object
The encoder never writes an object without records, but decode_footer
accepted a footer that declares zero segments, and
verify_segment_ranges accepts an empty footer too. Only the test-only
decode_object rejected it, so a checksum-valid empty object would pass
the header, trailer and footer checks that recovery runs.
Reject a zero entry count in decode_footer and drop the now unreachable
check in decode_object. Add a test that builds a checksum-valid object
with an empty footer and checks that decode_footer and decode_object
reject it.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* refactor(log-store): use pub(crate) for the WAL object format API
Other log-store modules use pub(crate) for items shared across module
boundaries. Switch the format module from pub(super) to pub(crate) to
follow that convention. No behavior change.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* feat: use series indexes for SeriesScan candidate discovery
Signed-off-by: evenyag <realevenyag@gmail.com>
* perf: use range indexes in two-phase series reader
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix(mito2): update series index test fixtures after rebase
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor(mito2): move lazy range index searchers into file contexts
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: pin series index snapshot before data snapshot
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: preserve builder caching for index-covered SSTs
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
DataFusion 55 deprecated ExecutionPlan::partition_statistics in favor of
statistics_from_inputs with StatisticsContext::compute. Migrate the two
remaining GreptimeDB overrides, RegionScanExec and MergeScanExec, and
update the RegionScanExec test to the new API, dropping its
allow(deprecated).
Behavior is unchanged: RegionScanExec keeps the append-mode-only,
exact-source-rows statistics gate from #9154, and MergeScanExec keeps
reporting unknown statistics for per-partition requests.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(servers): prototype ready-only Flight batch coalescing
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(servers): bound ready Flight batch admission and cover lifecycle
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(servers): fix redundant error pattern assertion
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(servers): simplify Flight coalescing with append-then-check accumulator
Remove the lookahead/pending machinery and coalesce with an
append-then-check soft budget. Extract the group state into a private
BatchAccumulator so the ordinary stream loop reads as a short control
flow: append each ready batch, flush when a budget is reached.
Budgets are flush thresholds, not memory limits, so a group may exceed
a budget by the final appended batch. First-batch direct send and
singleton passthrough for a batch that starts a group already at a
budget are unchanged.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(servers): split Flight stream into dispatcher and per-path units
FlightRecordBatchStream::flight_data_stream held two large side-by-side
branches (the verbose/ANALYZE metrics path and the plain/coalescing
path) inline, making the function hard to read. Split it into a small
dispatcher that keeps only the shared prologue (schema send, metrics
init) and the shared EOF final-metrics tail, and move each branch into a
self-contained unit:
- verbose path -> Self::verbose_metrics_stream
- plain/coalescing path -> private CoalescingBatcher (owns the
BatchAccumulator, sent_first_batch, and schema)
Each path returns whether it reached normal EOF; the dispatcher skips
the shared final-metrics tail on any early error/failed-send exit,
preserving the exact pre-split behavior (an early return previously
exited the whole function and bypassed the tail). No behavior, metrics
ordering, or coalescing semantics change.
Add a regression test that drives flight_data_stream directly and
asserts the verbose error path does not invoke the final-metrics tail
(producer-side metrics() call count), since the public message stream
hides the tail after an error.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* docs(servers): describe Flight stream units by behavior, not refactor history
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(servers): avoid copying over-budget batches and release merged inputs
Three review-driven improvements to Flight batch coalescing:
- After a successful merge, reuse the drained vector in place
(clear + push the merged batch) so the source batches are dropped
before the send loop instead of staying alive across a backpressured
send.
- A batch that is itself at/over a budget is forwarded as a singleton
even when encountered inside an accumulation group: the accumulated
group is flushed and sent first, then the over-budget batch. This
avoids copying a large batch into an aggregate just to merge the small
batches ahead of it. Under-budget append-then-check is unchanged
(600+600 still merges).
- Strengthen two tests: the first-batch test now drives the producer
directly and asserts the upstream poll count is exactly one at the
first-batch send; the dictionary test now decodes the merged
dictionary keys to assert the logical values, so a key-remapping
regression is caught.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(servers): raise Flight coalesce row budget to 4096
mito2 commonly emits ~2000-row batches (~32-94KiB), so a 1024-row budget
marked every such batch oversized and the coalescer forwarded them
unbatched. Measured on a local distributed cluster (2M-row table):
- full selector: 1000 -> 337 batches, p50 latency 4133 -> 3316 ms (-20%)
- range scan: 501 -> 206 batches, 1466 -> 1162 ms (-21%)
- small/olap queries (single-series, top-k, group-by): unchanged
Larger budgets (8192 rows, or 1-4MiB bytes) coalesce more batches but do
not improve latency further and slightly regress it, so 4096 rows is the
sweet spot; MAX_BYTES stays at 256KiB and MAX_BATCHES at 16.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* feat(wal): add the object store WAL provider identity and configuration
Add the identity and configuration of the experimental object store WAL
without the log store implementation:
- store-api: `Provider::ObjectStore` scoped by region id and prefix; it is a
remote WAL.
- common-wal: `DatanodeWalConfig::ObjectStore` (`experimental_object_store`)
with `storage_provider`, `prefix`, `flush_interval`, `max_batch_bytes` and
`on_corrupted_segment`, and `WalOptions::ObjectStore` persisted as
`object_store` with the key `wal.object_store.prefix`. The metasrv config
conversion rejects the new provider.
- common-meta, meta-srv, mito2: handle the new variants, map the region WAL
options to the provider and reject them on a Raft Engine or Kafka log store.
- datanode: validate the configuration and fail with a "not supported yet"
error until the log store lands.
- Example configs and the generated config docs.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(wal): accept object store WAL options when re-enabling WAL
Count object store WAL options as an existing WAL provider when setting skip_wal to false, drop an inaccurate replay note on Provider::is_remote_wal and fix the new rustdoc link.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* perf(promql): reuse sliding min and max candidates
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(promql): simplify extrema benchmark parameters
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(promql): record baseline sliding extrema SQL results
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(promql): rescan windows that barely overlap
Reusing candidates loses to a plain scan when consecutive windows overlap
little: the deque bookkeeping then costs more than the rescan it replaces.
A local Criterion run on 4096 samples at width 240 / step 240 measured
10.79 -> 22.14 us for min and 12.83 -> 19.85 us for max.
Pick the evaluator once per batch from the first two windows. RangeManipulate
emits one window length and one step per batch, so that sample decides for all
of them, and both evaluators return identical bits, so a wrong pick costs time
only. Batches that do not qualify fold each window on its own.
Move the incremental state into SlidingExtrema so tests can drive it directly:
the exhaustive four-sample differential test cannot reach it through a UDF
call, because such a batch never qualifies for reuse.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
* fix(promql): select the extrema evaluator from batch averages
Reading the window shape off the first two windows misreads the batch.
RangeManipulate starts a series at max(query start, first aligned sample),
so a series that begins inside the query range gets a first window covering
roughly one step, and a window covering no sample at all is emitted as
(0, 0). Either one closed the gate for the whole batch, including the
one-hour window at a 15s step that candidate reuse was written for.
Compare the batch averages instead: at least 32 samples per window, and a
step advancing at most a quarter of that. Uniform batches select exactly as
before, so the thresholds keep the meaning they were measured with.
The 32-sample rule had also moved most of the benchmark and query-regression
shapes onto the rescan, including the case built to measure reset and
rebuild. Widen those windows to 40 samples, add a step at the selection
boundary, and add an end-to-end case with 40-sample windows advancing 5.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
* fix(promql): ignore empty windows when measuring batch advance
The advance was read from the first and last window offsets, but a window
covering no sample is emitted as (0, 0). A query whose last evaluation lands
exactly one window past the last sample ends on such a window, and its zero
offset made a batch of disjoint windows look like one that never moved, which
selected the evaluator built for overlap. Results stayed correct; the cost was
deque bookkeeping on the shape the scan fallback exists for.
Take the offset span over the windows that cover a sample. Empty windows stay
in the window count, where they only make both conditions stricter.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
Co-authored-by: Dennis Zhuang <xzhuang@greptime.com>
* fix(mito2): preserve last-non-null values during compaction
Use a dedicated LastNonNullPicker to close TWCS seeds over overlapping SSTs, merge connected seed groups, and defer closures with busy inputs. Keep LastRow and strict-window selection unchanged.
Index overlap candidates on the blocking compaction runtime and cover field promotion, transitive dependencies, expiration, and conservative key-range pruning with regression tests.
Refs #9146
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): guard last-non-null compaction against pending memtables
Track conservative memtable sequence lower bounds and capture them from the same version as compaction inputs. Defer unsafe SST closures and strict-window plans until pending writes no longer intersect their sequence span.
Cache minimum sequences during bulk conversion and expose a lightweight memtable getter to avoid rescanning rows or computing full statistics.
Fixes#9157
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): limit eligible last-non-null compaction outputs
Enumerate all TWCS seeds for LastNonNull while retaining bounded window planning concurrency. Apply the output budget after closure expansion and busy/memtable safety checks, preserving execution priority and ordinary TWCS behavior.
Cover rejected and absorbed seeds, priority ordering, and bounded and unlimited output counts with real-picker regression tests.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): include all candidates before last-non-null closure
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito2): clarify last-non-null compaction closure
Extract transitive closure expansion from output scheduling and clarify seed handling. Document the dedicated picker rationale and schema-evolution risks in PK overlap checks.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test(mito2): cover deferred non-seed compaction bridges
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): defer strict-window compaction with busy dependencies
Keep busy SSTs when building the LastNonNull strict-window dependency closure, then defer the entire rewrite plan if any selected input is busy. Preserve LastRow filtering and expired-file cleanup.\n\nCover requested, transitive and unrelated busy windows, including retry after the dependency is released.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs(mito2): document compaction overlap index fields
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
dennis zhuang
jeremyhi
Yingwen
Mohd Quamar Tyagianddennis zhuang
Dhruv VaishnavandWeny Xu
Sainath Singineedi
discord9anddiscord9
Lei, HUANG
LFC
fys
shuiyisong
