* perf(telemetry): remove the shared lock from TraceLayer
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(telemetry): stop collecting span data while tracing is disabled
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): keep value-field grouping labels out of matching filter propagation
#9202 propagates matching-label matchers to the scanned selector using the
planned operands' tag columns. For an aggregated operand those are its
grouping labels, and agg_modifier_to_col resolves by(...) names against the
input schema only, so a value field named in by(...) is reported as a tag.
A value field varies between the samples of one series, so lowering a
matcher on it below sample selection (PromInstantManipulate) can drop the
newest sample, promote a stale one from the lookback window, and fabricate
a match the un-rewritten query does not produce.
Track the by(...) labels that name value fields of the aggregated operand's
input in PromPlannerContext::aggregation_field_labels, and refuse to
propagate matchers on them while still requiring a matcher to name a
grouping label to cross an aggregation.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test: sync matching_filter result fixture comment with the PR number
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(servers): stream Prometheus HTTP responses incrementally
Replace try_collect + full-buffer conversion in the Stream branch of
from_query_result with incremental per-batch merging via the shared
merge_batch path, so record batches are dropped as soon as they are
consumed instead of all coexisting in memory.
- Add ColumnLayout to dedupe schema inference shared by both paths
- Add SeriesKeyLookup (Equivalent-based borrow lookup) so series keys
are looked up by borrowed label slices without materializing owned
keys per lookup
- Empty streams short-circuit to empty data before schema inference
- Add stream-side mixed histogram + Vector dual-path equivalence test
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(servers): address review comments on stream response tests
- Inline single-caller tags_capacity helper
- Drop timestamp column from empty-stream test so it pins the
short-circuit happens before schema inference
- Fix cross-batch merge test data so a series actually spans the
batch boundary (second batch carries earlier timestamps)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(servers): add lazy stream release assertion and pinned cross-batch content
- LazyDropCheckingStream creates one batch per poll and asserts via
Weak that the previous batch was released before the next poll,
protecting the core property that batches are dropped eagerly
- Pin series "a" merged samples to exact timestamps and values in
stream_result_matches_record_batches_result, so a regression in
shared merge_batch logic cannot make both paths agree on wrong output
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): insert MergeScan into nested scalar subqueries
DataFusion 55 keeps uncorrelated scalar subqueries as expression
subqueries (enable_physical_uncorrelated_scalar_subquery, default true)
instead of decorrelating them into joins, and executes them via the new
physical ScalarSubqueryExec. DistPlannerAnalyzer::try_push_down walked
the plan with a plain TreeNode transform that does not descend into
expression subqueries, so MergeScan was only inserted for depth-1
subqueries. A scalar subquery nested inside another scalar subquery kept
a bare frontend DistTable TableScan and failed at execution with
"Unsupported operation: get stream from a distributed table".
Use the subquery-aware transform so handle_subquery (PlanRewriter /
MergeScan insertion) runs for subquery plans at every nesting depth.
Fixes#9260.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(query): strengthen nested scalar subquery regression coverage
Address review on #9261:
- Replace the ineffective 'no bare TableScan' string check with a real
subquery-aware plan walk (apply_with_subqueries); MergeScan hides its
remote input from traversal, so any TableScan the walk reaches was
genuinely left unwrapped.
- Add a distributed regression case on a range-partitioned table so the
nested inner aggregate must merge partial results across regions
(global AVG feeding an outer SUM filter).
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(mito2): compare primary key ranges across schema versions
Bind FileHandle ranges to the pinned region schema and append cached constant defaults to historical Dense keys. Preserve raw SST statistics, reject inexact bounds, and avoid invalidating views for unrelated metadata changes.
Cover schema evolution, default changes, and tombstone retention through real compaction and reopen regressions.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito2): report invalid primary key ranges
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito2): scope primary key ranges to comparisons
Keep raw PK bounds only in FileHandleInner and move schema-aware mapping and caching into task-local comparison contexts.
Use explicit contexts for compaction overlap checks, window aggregation, and series scans. Preserve pinned-schema isolation, late statistics, and shared file lifecycle state without rebinding every handle.
Cover cache isolation across region owners and adapt range fixtures to real Dense encodings. All 1530 mito2 tests and Clippy for all targets with the testing feature pass.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito2): cache aligned primary key ranges per file
Replace task-local range maps with a single-slot cache in FileHandleInner, keyed by the target schema version. Preserve raw bounds for realignment across snapshots and default changes.
Share schema mappers across comparison paths and use copy-on-write SST lists for metadata updates. Simplify range mapping to accept encoded bounds and assert the same-table contract at the file accessor.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs(mito2): clarify primary key mapper schema snapshot
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test(mito2): align primary key range fixtures with table contract
Remove obsolete cross-table fallback expectations after region validation became a caller contract. Give compaction fixtures matching table identities, including the active-window L1 scenario.
Clarify the mapper precondition and format the simplified alignment call. All 1530 mito2 tests and Clippy for all targets with the testing feature pass.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test(mito2): enable filesystem GC in release unit tests
Let unit tests use the filesystem-backed object-store GC path regardless of optimization profile. Keep the production release GC selection unchanged.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* perf(mito-codec): skip release value decoding in PK prefix counts
Validate field values only in debug builds and unit tests while keeping boundary, truncation, and trailing-byte checks in every build.
Cover the linked library in debug and release integration tests, and verify that release unit tests still perform value validation.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test(mito-codec): remove redundant prefix integration tests
Retain the codec unit tests and cross-schema compaction regressions while dropping the standalone build-profile test file and its release-only invalid-value expectation.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* ci: wait for MySQL to accept authenticated TCP queries
Add a healthcheck using the configured test account and database. Docker Compose --wait previously only observed container startup because the fixture image had no healthcheck, allowing metasrv to connect before MySQL initialization completed.
Verify readiness with SELECT 1 over TCP rather than the initialization socket.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* perf(prom): release the remote write v1 decode buffer before writing
remote_write_v1 kept the decoded builder alive until the handler returned,
so the decompressed request payload stayed resident across the downstream
write or pipeline await. With 8 concurrent large requests that is one extra
copy of every payload held for the whole write.
Rows and pipeline values own their data, so the builder can be dropped as
soon as the conversion is done.
Sustained-write A/B, 8 runs per side, 50M samples each: jemalloc allocated
median drops 7.8% (155.0-162.7 MiB -> 141.4-158.6 MiB); samples per CPU
second is unchanged (-0.6%, fully overlapping ranges).
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* perf(servers): move row values into SQL JSON responses
The format=json renderer cloned every serde_json::Value and kept the whole
row set alive while building the response. Move the values out instead, so
each row is released as soon as it is converted.
Slicing the row to the schema width keeps the panic on rows narrower than
the schema; a plain zip would silently truncate them. Duplicate column names
still resolve to the last value and extra row values are still ignored.
Isolated conversion measurements: live peak drops 33% on a 4096-row 16 KiB
string fixture and 36% on a nested-JSON fixture, with no measured slowdown.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* perf(prom): release the compressed remote write body after decompression
Both the v1 and v2 decoders held the compressed Bytes until they returned,
which spans the whole protobuf decode and row conversion. Decompression
copies the payload into an independent buffer, so the body can go as soon
as it succeeds.
The compression fallback, decode errors and request counting are unchanged.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* revert(prom): keep the remote write v1 decode buffer until the write finishes
This reverts commit 8232c5b3bb.
The v1 decoder fabricates `&'static [u8]` pointing into its own decode buffer
(prom_remote_write/types.rs), so the compiler checks nothing about that
buffer's lifetime. Holding the builder until the handler returns is what keeps
the decoder safe by construction; releasing it early made that safety depend on
every consumer copying out of the buffer, which holds today but nothing
enforces.
Document the requirement at the binding instead.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* perf(prom): release the remote write v1 decode buffer before writing
This reverts commit ca3af722d0, restoring
8232c5b3bb.
The decoder borrows the decompressed buffer while parsing, but copies
everything out when it builds rows: tag values through
`PromValidationMode::decode_string`, column names through `to_owned`, and the
only live borrows (`TableBuilder::col_indexes`) are dropped inside
`as_insert_requests`. The resulting `ContextReq` holds prost types with no
lifetime parameters, so it cannot reference the buffer.
Record that at the binding so the next reader does not have to re-derive it
from three files.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(mito2): split SWCS output files by size threshold
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): use resolved SWCS output size options
Read the SWCS output file size threshold from the resolved region options so database-level compaction settings are honored. Align the existing picker test with this configuration source.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: prefer JSON2 type hints for uncast read pushdown
* feat(query): materialize JSON get result types before planning
* fix(query): apply JSON2 type hints to parsed paths
* fix(query): apply JSON2 type hints before distributed planning
* chore: remove f
* refactor: the code style of json_get_type_hint
* chore: add sqlness case
* fix: add json expr planner, remove json get type hint
* chore: add sqlness test
* fix(query): respect JSON2 type hints in query planning
* chore: add more sqlness cases
* fix: cargo clippy
* fix: cargo clippy
* chore: update sqlness test result
* perf(otlp): share trace resource and scope attributes
Parsing an OTLP trace request copied the resource attributes and the
scope attributes once per span. A request with wide resource attributes
kept one full copy per span alive until the rows were built.
Spans and their group now share one `Arc` per resource and per scope.
Empty attributes stay `None`, so a resource or scope without attributes
costs no allocation and no refcounting. v0 takes ownership back when it
encodes, v1 clones attributes item by item instead of rebuilding a Vec,
and v2 borrows the span instead of cloning it for every row.
Column values, attribute order, the `service.name` lookup and the
auxiliary table writes are unchanged.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* perf(otlp): stop copying v1 attribute keys
The v1 row writer rebuilds every column name as `{prefix}.{key}`, so the
key string it cloned from the shared resource and scope attributes was
dropped unused. `resource_attributes.service.name` was also cloned in
full before being skipped for the top level service name column.
Shared attributes are now read in place and only values that reach a row
are copied. Span attributes keep moving their values as before.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(mito2): cancel cache construction for incomplete scans
CacheBatchBuffer spawned the background concat task and dropped its join
handle. A scan that was cancelled or failed therefore left the task alive:
it kept compacting already queued batches, and while waiting for a range
result memory permit it held them, even though without a finish command
the result can never be put into the cache.
Keep the handle and abort it when the buffer is dropped. The handle is
cleared once the task owns the finish command, so a completed scan still
populates the cache after its stream is dropped. Abort does not preempt a
concat that is already running; it takes effect the next time the task is
polled.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(mito2): wait for the permit park before cancelling the buffer
An empty buffered_batches only proves the batches were enqueued, so the
cancellation test could abort a concat task that had never been polled.
Count acquisitions that find too few permits, a test-only signal, and drop
the buffer once the task has reached that wait. Nothing awaits between the
check and the parking, so an observed increment means the caller is about
to wait for permits the test holds for the rest of the case.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: add config flag to control range index reads
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: disable range index builds when configured and default to off
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
CompactDispatcher acquired a borrowed permit inside the async wrapper and
dropped it as soon as the blocking task was submitted, so the semaphore
never bounded the compactions that actually ran. Acquire an owned permit
and move it into the blocking closure so it covers the queue wait and the
merge itself.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(prometheus): honor label matchers in __name__ values query
`/api/v1/label/__name__/values?match[]={pod="abc"}` dropped every matcher
other than `__name__` and returned all metrics in the schema. No error,
just the wrong list. Grafana's metrics browser sends this request, so
picking a label value there did nothing.
Selectors that only constrain `__name__` keep answering from table
metadata. A selector constraining an ordinary label now goes to the data:
scan each metric engine physical table for distinct `__table_id` in the
time range, map the ids back to metric names, then apply the selector's
own `__name__` matchers.
Only metric engine tables are covered; other engines share no column space
to scan.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(prometheus): batch-resolve metric names by table id
Building a full table-id-to-name map meant walking every table in the
schema and holding all of them in memory, just to name the handful the
scan returned. Use `tables_by_ids` instead — one batch KV read over the
ids the scan actually produced.
The catalog walk stays, but only to find the physical tables to scan.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): read an absent label as the empty string
A matcher on a label the series does not carry only worked when the table
had no column for it at all. Where the column exists but is NULL on that
row -- the norm for logical metrics sharing a metric engine physical
table, which holds the union of their label columns -- three-valued logic
dropped the row, so `host!="host1"` and `host=""` missed every metric
without a host label.
Coalesce nullable string label columns to "" for matchers that accept the
empty string, rather than only for the OTLP temporality marker. Equality
matchers are untouched; they cannot match NULL either way.
This is the Prometheus compatibility fix#8970 deliberately kept out of
its own scope. The cost is visible in the regex sqlness plan: the
predicate becomes a CASE, so the scan loses its LastRow selector and
grows a FilterExec. Only negative and empty-accepting matchers pay it.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): don't panic on pre-epoch label value bounds
`rewrite_label_values_query` unwrapped `duration_since(UNIX_EPOCH)`, which
returns an error for an instant before the epoch. `start=1969-12-31T23:59:59Z`
parses as valid RFC3339, so the request panicked instead of answering.
Recover the sign from the error branch, and report a value beyond i64
milliseconds as an error rather than wrapping the cast.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(prometheus): drop applicable_matchers, share the distinct scan
With the planner reading an absent label as empty, the frontend no longer
needs to pre-filter matchers per physical table. Removing that exposed a
second problem: a physical table that never took a column from a logical
table exposes no `__table_id`, and projecting it failed the whole request.
Skip those tables; the only thing that can miss is a metric with no labels.
Also pulls out the plan-build-execute-collect sequence the two label value
scans had in common.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Incremental batching flows already merge sink state for scalar
aggregates and the HLL/UDDSketch/stddev state families. AVG now exposes
a mergeable Binary state (avg_state / avg_merge with
__avg_state_delta_merge), so admit those aggregates too instead of
forcing a full snapshot for flows whose only state column is an
average.
merge_op_for_aggregate_expr takes the aggregate input schema so the
avg_merge arm can require a Binary state argument; the state form is
the aggregate result persisted by the sink, so no extra coercion is
needed. Other input types keep being rejected.
Also cover avg_state, avg_merge and duplicate AVG projections in the
incremental plan analysis tests, extend the mixed state-family rewrite
test with an average column, and extend the standalone partitioned
state-merge SQLness case with avg_state/avg_merge compared against a
direct avg over the source.
Signed-off-by: discord9 <discord9@outlook.com>
Co-authored-by: discord9 <discord9@outlook.com>
* perf(promql): propagate matching-label filters between binary operands
A one-to-one arithmetic binary expression inner-joins its operands on the
matching labels, so every row that survives the join already satisfies the
other operand's equality matchers on those labels. Copy those matchers to the
other operand so both scans drop non-joining series before execution instead of
feeding them to the join.
Both operands are planned before the rewrite: a selector matcher can also
constrain a value field, and only the planned contexts tell tags and fields
apart. A matcher is copied only when its name is a tag column on both sides, its
value is non-empty, and it is one of the matching labels. Operands are limited
to vector selectors, parentheses, and label-preserving range functions applied
directly to a matrix selector; `ignoring(...)`, group modifiers, fill values,
set and comparison operators, regex matchers and `or` matcher groups are left
alone.
Selector matchers are now deduplicated in place instead of through a `HashSet`,
so the generated scan filter keeps a stable order once a selector carries more
than one matcher.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* perf(promql): propagate ignoring, non-equality and aggregated matchers
Widens the matcher propagation added in the previous commit to the shapes it
was leaving on the table.
The join compares matching labels with plain column equality
(`normalized_match_key_expr` and its coalescing are confined to `or`), so any
predicate on a matching label is already enforced on both sides for every
surviving pair: it originates on one operand, and the join carries it to the
pairs it forms. Copying it to the other operand can only drop rows that had no
surviving partner. That argument does not depend on the matcher kind, so regular
expressions, negations and empty values now propagate too. `=~".*"` stays out:
it lowers to no filter at all, so copying it would only force a re-plan.
`ignoring(...)` is no longer rejected. A label is a join key exactly when it is a
tag on both sides and not named in `ignoring`, which is what
`binary_join_key_columns` computes and what the caller can now answer from the
two planned contexts.
Operands may now be aggregations that partition by their grouping labels, which
is the shape most real queries use. `agg_modifier_to_col` rewrites
`ctx.tag_columns` to the grouping labels, so a label found in an aggregated
operand's tags is a group key, and filtering the aggregate's input by it drops
exactly the corresponding output groups. `topk`, `bottomk` and `limitk` are
excluded because they select across a group and carry input labels through --
`prom_topk_bottomk_to_plan` leaves `ctx.tag_columns` alone, so the tag check
cannot catch them. `count_values` is excluded because it adds an output label
that does not exist in its input.
The rollup whitelist now covers every label-preserving range function the
planner implements, and finds the matrix argument by position so multi-argument
rollups such as `quantile_over_time` and `predict_linear` qualify.
`absent_over_time` stays out: it synthesizes a series from the matchers when its
input has none.
Re-running the sqlness case against an unmodified planner produces a
byte-identical `.result` for all 27 queries.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): pin that excluded matching labels stay on their own operand
`ignoring(device)` and `on(host)` both leave `device` out of the join keys, so a
`device` matcher must not reach the other operand. Neither case was covered end
to end: a regression there silently drops the left operand's `eth1` series
instead of returning them, which the two added queries now catch.
Also corrects the comment at the rewrite site, which still described re-planning
as touching a leaf selector. Aggregated operands are re-planned as a whole; what
holds is that the rewrite only ever adds matchers to a selector, so the enclosing
operand's table reference, time index and field columns are unchanged.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(promql): log why matching-filter propagation was skipped
Splits the guard chain into `try_propagate`, whose `Err` names the reason the
rewrite does not apply, and logs it once in `propagate`. Debugging a query that
did not get the filter no longer means stepping through the guards.
The reasons also replace the comments that used to explain the same conditions,
and the remaining comments lose the parts that restated the code or repeated
each other.
Two of them were wrong rather than verbose. Saying `topk`'s input "must not be
filtered" reads as a claim about PromQL: `topk(1, m{host="x"})` is perfectly
legal, and what matters is that filtering before `topk` changes the candidate
set it ranks. Saying the subset-matching case "keeps its many-to-many result"
read as an endorsement of behaviour Prometheus rejects outright; it now records
that the query is a cross product here and points at #9209.
The propagation cannot change whether such a check would fire: series in one
match group agree on every matching label, so a matcher over one of those labels
keeps all of them or drops all of them.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): keep copied matchers out of the operand's selector metadata
Re-planning an operand with a copied matcher also rebuilt its
`PromPlannerContext::selector_matcher`, and that context is what an enclosing
expression reads. `create_absent_plan` turns the equality matchers found there
into the labels `absent()` reports, so
absent(counter_metric{host="missing"} / on(host, device) gauge_metric)
gained a `host="missing"` label that `main` does not produce. The inner
expression was empty either way; only the reported label set changed.
The copied matcher belongs to the scan, not to the operand's identity, so both
re-planned contexts now keep the matchers their operand was written with.
`selector_matcher` has one other reader, `create_table_scan_plan`, which
consumes it while the operand is being planned and is unaffected.
Reported by @discord9.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(log-store): add the object store WAL batch, catalog and I/O
Add the three modules between the object format and the store of the
object store WAL:
- batch: the open batch that accumulates admitted entries into the next
object and assigns entry ids at admission. Ids are object-sequence-major,
`(object_seq << 20) | position`, with positions starting at one per
region per object, so a batch that is rolled back and admitted again
under the same sequence hands out the same ids. The time the first entry
was admitted is kept for the store's age-based sealing; an empty
admission does not start it.
- catalog: the in-memory index over object footers, by sequence and per
region. Insertion is atomic and rejects an empty footer, duplicate
region segments, invalid entry ranges, an already indexed sequence and
entry ranges that are not strictly increasing across objects. The next
object sequence continues after the largest indexed one and is raised
above the largest entry id of any region, and is rejected once it no
longer fits an entry id.
- io: object store access under `<prefix>/objects/`: a conditional create
whose retry with identical content is a no-op and whose conflicting
content is rejected, whole and range reads, and a listing of well-formed
keys in sequence order. A prefix with a `.` or `..` component is
rejected. When the store reports that the object exists but the read
that compares its content fails, the read failure is returned with its
retry hint instead of the create failure.
The modules have no callers until the store lands, so they are declared
with `#[allow(dead_code)]`. Only `entry_id` is exported.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(log-store): drop comments that restate the batch test assertions
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* refactor(log-store): use pub(crate) in the object store WAL batch, catalog and I/O
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(log-store): keep entry_id crate-private and narrow the create collision check
Re-export `entry_id` as `pub(crate)`: nothing outside `log-store` uses it
yet, and exporting it would freeze the raw `(object_seq, position)`
encoding before the store owns id allocation.
Treat only `ConditionNotMatch` as the sign that a conditional create
collided with an existing object. A store may report `AlreadyExists` for
an unrelated path, for example when a parent of the object is a file, and
that failure must come back as the write failure rather than as the error
of the read that would compare content.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* refactor(log-store): rename the catalog's out_of_order helper to out_of_order_reason
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* feat(function): add mergeable binary average states
Signed-off-by: discord9 <discord9@163.com>
* feat(function): expose avg_calc as an OSS scalar
Signed-off-by: discord9 <discord9@163.com>
* fix(function): borrow invalid AVG scalar argument type
Signed-off-by: discord9 <discord9@163.com>
* test(function): verify public AVG state SQL finalization
Signed-off-by: discord9 <discord9@163.com>
* fix(function): return canonical AVG1 state for empty window frames
DataFusion's plain-aggregate window executor bypasses the accumulator
and calls default_value() directly when a window frame contains no
rows. create_udaf's SimpleAggregateUDF derives default_value from the
return type, which yields SQL NULL for Binary output instead of the
canonical AVG1 empty state, so empty frames and frames over only NULL
inputs became observable differently (IS NULL, direct state saves,
state comparisons).
Register avg_state/avg_merge through a small AggregateUDFImpl (AvgUdaf)
that keeps the existing accumulator and declares the canonical AVG1
empty state as default_value, restoring the documented contract. Add a
unit test asserting default_value equals the empty accumulator's
evaluate() and an sqlness case covering the empty-frame window.
Signed-off-by: discord9 <discord9@outlook.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@outlook.com>
Co-authored-by: discord9 <discord9@outlook.com>
* chore: bump memcomparable to d8fb3558 for bytes I/O optimizations
The new revision (v0y4g3r/memcomparable main, PR #1) includes:
- Deserializer::read_bytes_into for reusable-buffer decoding
- read_bytes 32B pre-allocation (no more 0→8→16→32 realloc chain)
- bulk put_slice in both normal and reverse modes
- Error::Eof instead of panic on truncated input
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* perf(mito2): extract sparse tag values lazily without full PK decode
Replace the full primary-key decode in two hot consumers of
SortField::deserialize with column-lazy extraction over
SparsePrimaryKeyView, eliminating the per-label owned values
(Vec<Value>, one heap allocation per string) for every distinct key:
- flat_format: DecodedPrimaryKeys keeps sparse keys encoded and
extracts each tag column directly from the raw dictionary bytes into
a reusable buffer. FlatConvertFormat (flush/compaction write path)
builds all projected tag columns in one pass per key, sharing offset
discovery between columns; the read path (file_range filters) keeps
per-column lazy extraction.
- series_index writer: builds rows from the reserved prefix and only
the indexed tags via a shared offsets cache and buffer, instead of
decoding all labels and copying out a few strings.
- mito-codec: add SparsePrimaryKeyView::reserved_value for typed
table_id/tsid extraction.
Persisted formats are unchanged; values pushed to column builders are
identical (NULL vs empty string, reserved column types, duplicate
dictionary entries and consecutive-run order are preserved).
Benchmark (bench_pk_tag_column, 4096 rows, 40 labels per key, Criterion
30 samples): old = full decode + per-column builds, new = lazy
extraction (+ one-pass for multi-column projection).
| Workload | Before | After | Change |
| --- | ---: | ---: | ---: |
| 1 of 40 tags, 4096 distinct keys | 6.95 ms | 803 us | -88% |
| 1 of 40 tags, 128 distinct keys | 206.8 us | 29.2 us | -86% |
| 10 of 40 tags, 128 distinct keys | 236.2 us | 75.5 us | -68% |
| 40 of 40 tags, 128 distinct keys | 314 us | 215.2 us | -31% |
| 1 of 10 tags, 4096 distinct keys | 1.75 ms | 207.6 us | -88% |
Also adds a decode_sparse benchmark group to mito-codec's
bench_sparse_encoding covering 10/40 labels x 10/24/64-byte values.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): share sparse tag scans across filters
Batch missing tag columns used by simple filters and partition pruning, while retaining single-column extraction and skip-mode semantics.
Replace the optional reserved-byte accessor with typed table_id/tsid reads from the validated prefix. Add full-decode equivalence tests for sliced dictionaries, NULL/empty labels and cross-key scratch reuse, plus series-index roundtrips across batches.
Benchmark real precise filtering with 4096 rows: 10 tag predicates drop from 307.85 to 162.49 us; 40 predicates drop from 894.03 to 550.55 us. All 1459 mito2/mito-codec tests pass.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test(mito2): benchmark sparse tag predicate count scaling
Add a dedicated precise-filter benchmark group for 1, 2, 4, 8, 16, and 32 predicates with both 1 and 32 rows per primary key. Keep 4096 rows, 40 labels, and full selectivity constant across cases, and validate the workload outside the timed section.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat(log-store): add the object store WAL object format
Add the byte format of a single object store WAL object: a header with
the GTWALOBJ magic, format version 1, the object sequence and the writer
instance id; one segment per region ordered by region id with entries
ordered by entry id; a footer that records each segment's region id,
entry id range, entry count, byte range and CRC32; and a fixed trailer
with the GTWALTRL magic, the footer location, the footer CRC32 and the
whole-object CRC32.
The module encodes objects deterministically and decodes the header,
trailer, footer and segments separately, with structural checks on
footer ranges and segment tiling. It has no callers yet; the store that
writes and reads objects follows in later changes.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(log-store): pin the object store WAL format with a byte fixture
Add a fixed version 1 object with two regions and five entries as a hex
literal. The test decodes it and checks the exact header, trailer,
footer entries and records, and checks that encoding the same records,
in either input order, reproduces the fixture byte for byte.
Round-trip tests alone pass when a refactor changes field order,
endianness or checksum coverage in both the encoder and the decoder.
The fixture bytes were derived from the documented layout rather than
from the encoder, so such a change now fails.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(log-store): reject an empty footer when decoding a WAL object
The encoder never writes an object without records, but decode_footer
accepted a footer that declares zero segments, and
verify_segment_ranges accepts an empty footer too. Only the test-only
decode_object rejected it, so a checksum-valid empty object would pass
the header, trailer and footer checks that recovery runs.
Reject a zero entry count in decode_footer and drop the now unreachable
check in decode_object. Add a test that builds a checksum-valid object
with an empty footer and checks that decode_footer and decode_object
reject it.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* refactor(log-store): use pub(crate) for the WAL object format API
Other log-store modules use pub(crate) for items shared across module
boundaries. Switch the format module from pub(super) to pub(crate) to
follow that convention. No behavior change.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* feat: use series indexes for SeriesScan candidate discovery
Signed-off-by: evenyag <realevenyag@gmail.com>
* perf: use range indexes in two-phase series reader
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix(mito2): update series index test fixtures after rebase
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor(mito2): move lazy range index searchers into file contexts
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: pin series index snapshot before data snapshot
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: preserve builder caching for index-covered SSTs
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
DataFusion 55 deprecated ExecutionPlan::partition_statistics in favor of
statistics_from_inputs with StatisticsContext::compute. Migrate the two
remaining GreptimeDB overrides, RegionScanExec and MergeScanExec, and
update the RegionScanExec test to the new API, dropping its
allow(deprecated).
Behavior is unchanged: RegionScanExec keeps the append-mode-only,
exact-source-rows statistics gate from #9154, and MergeScanExec keeps
reporting unknown statistics for per-partition requests.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(servers): prototype ready-only Flight batch coalescing
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(servers): bound ready Flight batch admission and cover lifecycle
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(servers): fix redundant error pattern assertion
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(servers): simplify Flight coalescing with append-then-check accumulator
Remove the lookahead/pending machinery and coalesce with an
append-then-check soft budget. Extract the group state into a private
BatchAccumulator so the ordinary stream loop reads as a short control
flow: append each ready batch, flush when a budget is reached.
Budgets are flush thresholds, not memory limits, so a group may exceed
a budget by the final appended batch. First-batch direct send and
singleton passthrough for a batch that starts a group already at a
budget are unchanged.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(servers): split Flight stream into dispatcher and per-path units
FlightRecordBatchStream::flight_data_stream held two large side-by-side
branches (the verbose/ANALYZE metrics path and the plain/coalescing
path) inline, making the function hard to read. Split it into a small
dispatcher that keeps only the shared prologue (schema send, metrics
init) and the shared EOF final-metrics tail, and move each branch into a
self-contained unit:
- verbose path -> Self::verbose_metrics_stream
- plain/coalescing path -> private CoalescingBatcher (owns the
BatchAccumulator, sent_first_batch, and schema)
Each path returns whether it reached normal EOF; the dispatcher skips
the shared final-metrics tail on any early error/failed-send exit,
preserving the exact pre-split behavior (an early return previously
exited the whole function and bypassed the tail). No behavior, metrics
ordering, or coalescing semantics change.
Add a regression test that drives flight_data_stream directly and
asserts the verbose error path does not invoke the final-metrics tail
(producer-side metrics() call count), since the public message stream
hides the tail after an error.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* docs(servers): describe Flight stream units by behavior, not refactor history
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(servers): avoid copying over-budget batches and release merged inputs
Three review-driven improvements to Flight batch coalescing:
- After a successful merge, reuse the drained vector in place
(clear + push the merged batch) so the source batches are dropped
before the send loop instead of staying alive across a backpressured
send.
- A batch that is itself at/over a budget is forwarded as a singleton
even when encountered inside an accumulation group: the accumulated
group is flushed and sent first, then the over-budget batch. This
avoids copying a large batch into an aggregate just to merge the small
batches ahead of it. Under-budget append-then-check is unchanged
(600+600 still merges).
- Strengthen two tests: the first-batch test now drives the producer
directly and asserts the upstream poll count is exactly one at the
first-batch send; the dictionary test now decodes the merged
dictionary keys to assert the logical values, so a key-remapping
regression is caught.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(servers): raise Flight coalesce row budget to 4096
mito2 commonly emits ~2000-row batches (~32-94KiB), so a 1024-row budget
marked every such batch oversized and the coalescer forwarded them
unbatched. Measured on a local distributed cluster (2M-row table):
- full selector: 1000 -> 337 batches, p50 latency 4133 -> 3316 ms (-20%)
- range scan: 501 -> 206 batches, 1466 -> 1162 ms (-21%)
- small/olap queries (single-series, top-k, group-by): unchanged
Larger budgets (8192 rows, or 1-4MiB bytes) coalesce more batches but do
not improve latency further and slightly regress it, so 4096 rows is the
sweet spot; MAX_BYTES stays at 256KiB and MAX_BATCHES at 16.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* feat(wal): add the object store WAL provider identity and configuration
Add the identity and configuration of the experimental object store WAL
without the log store implementation:
- store-api: `Provider::ObjectStore` scoped by region id and prefix; it is a
remote WAL.
- common-wal: `DatanodeWalConfig::ObjectStore` (`experimental_object_store`)
with `storage_provider`, `prefix`, `flush_interval`, `max_batch_bytes` and
`on_corrupted_segment`, and `WalOptions::ObjectStore` persisted as
`object_store` with the key `wal.object_store.prefix`. The metasrv config
conversion rejects the new provider.
- common-meta, meta-srv, mito2: handle the new variants, map the region WAL
options to the provider and reject them on a Raft Engine or Kafka log store.
- datanode: validate the configuration and fail with a "not supported yet"
error until the log store lands.
- Example configs and the generated config docs.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* fix(wal): accept object store WAL options when re-enabling WAL
Count object store WAL options as an existing WAL provider when setting skip_wal to false, drop an inaccurate replay note on Provider::is_remote_wal and fix the new rustdoc link.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* perf(promql): reuse sliding min and max candidates
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(promql): simplify extrema benchmark parameters
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(promql): record baseline sliding extrema SQL results
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(promql): rescan windows that barely overlap
Reusing candidates loses to a plain scan when consecutive windows overlap
little: the deque bookkeeping then costs more than the rescan it replaces.
A local Criterion run on 4096 samples at width 240 / step 240 measured
10.79 -> 22.14 us for min and 12.83 -> 19.85 us for max.
Pick the evaluator once per batch from the first two windows. RangeManipulate
emits one window length and one step per batch, so that sample decides for all
of them, and both evaluators return identical bits, so a wrong pick costs time
only. Batches that do not qualify fold each window on its own.
Move the incremental state into SlidingExtrema so tests can drive it directly:
the exhaustive four-sample differential test cannot reach it through a UDF
call, because such a batch never qualifies for reuse.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
* fix(promql): select the extrema evaluator from batch averages
Reading the window shape off the first two windows misreads the batch.
RangeManipulate starts a series at max(query start, first aligned sample),
so a series that begins inside the query range gets a first window covering
roughly one step, and a window covering no sample at all is emitted as
(0, 0). Either one closed the gate for the whole batch, including the
one-hour window at a 15s step that candidate reuse was written for.
Compare the batch averages instead: at least 32 samples per window, and a
step advancing at most a quarter of that. Uniform batches select exactly as
before, so the thresholds keep the meaning they were measured with.
The 32-sample rule had also moved most of the benchmark and query-regression
shapes onto the rescan, including the case built to measure reset and
rebuild. Widen those windows to 40 samples, add a step at the selection
boundary, and add an end-to-end case with 40-sample windows advancing 5.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
* fix(promql): ignore empty windows when measuring batch advance
The advance was read from the first and last window offsets, but a window
covering no sample is emitted as (0, 0). A query whose last evaluation lands
exactly one window past the last sample ends on such a window, and its zero
offset made a batch of disjoint windows look like one that never moved, which
selected the evaluator built for overlap. Results stayed correct; the cost was
deque bookkeeping on the shape the scan fallback exists for.
Take the offset span over the windows that cover a sample. Empty windows stay
in the window count, where they only make both conditions stricter.
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
Co-authored-by: Dennis Zhuang <xzhuang@greptime.com>
* fix(mito2): preserve last-non-null values during compaction
Use a dedicated LastNonNullPicker to close TWCS seeds over overlapping SSTs, merge connected seed groups, and defer closures with busy inputs. Keep LastRow and strict-window selection unchanged.
Index overlap candidates on the blocking compaction runtime and cover field promotion, transitive dependencies, expiration, and conservative key-range pruning with regression tests.
Refs #9146
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): guard last-non-null compaction against pending memtables
Track conservative memtable sequence lower bounds and capture them from the same version as compaction inputs. Defer unsafe SST closures and strict-window plans until pending writes no longer intersect their sequence span.
Cache minimum sequences during bulk conversion and expose a lightweight memtable getter to avoid rescanning rows or computing full statistics.
Fixes#9157
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): limit eligible last-non-null compaction outputs
Enumerate all TWCS seeds for LastNonNull while retaining bounded window planning concurrency. Apply the output budget after closure expansion and busy/memtable safety checks, preserving execution priority and ordinary TWCS behavior.
Cover rejected and absorbed seeds, priority ordering, and bounded and unlimited output counts with real-picker regression tests.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): include all candidates before last-non-null closure
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* refactor(mito2): clarify last-non-null compaction closure
Extract transitive closure expansion from output scheduling and clarify seed handling. Document the dedicated picker rationale and schema-evolution risks in PK overlap checks.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test(mito2): cover deferred non-seed compaction bridges
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): defer strict-window compaction with busy dependencies
Keep busy SSTs when building the LastNonNull strict-window dependency closure, then defer the entire rewrite plan if any selected input is busy. Preserve LastRow filtering and expired-file cleanup.\n\nCover requested, transitive and unrelated busy windows, including retry after the dependency is released.
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* docs(mito2): document compaction overlap index fields
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* feat: allow re-enabling WAL after disabling
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
* test: regenerate skip_wal sqlness result
The previous commit changed tests/cases/standalone/common/skip_wal.sql without
regenerating the matching .result, so every Sqlness suite failed on the
mismatch. tests/cases/distributed/common is a symlink to standalone/common, so
the single stale file accounted for all five failing variants.
Regenerated from a real run. The recorded output now covers the cases the .sql
added:
* A table created with skip_wal = 'true' has no real WAL provider, so enabling
WAL is refused with Unsupported rather than the previous InvalidArguments.
* A RaftEngine-backed table accepts the true -> false transition, and repeating
it is a no-op.
* SHOW CREATE TABLE reports skip_wal = 'false' after the transition, including
after a restart.
* A row written while WAL was skipped is absent after restart, while a row
written after WAL was restored survives.
Verified locally with nothing else running on the machine: sqlness skip_wal
passes in both the standalone and distributed environments, and the regenerated
file is byte-identical to the output of an independent earlier run.
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
* test: strengthen skip_wal provider coverage
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
* Fix WAL provider handling during re-enable
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
* Refactor WAL provider check
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
---------
Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
dennis zhuang
discord9
Weny Xu
Ning Sun
Lei, HUANG
LFC
fys
Yingwen
Mohd Quamar Tyagianddennis zhuang
Dhruv VaishnavandWeny Xu
Sainath Singineedi
jeremyhi
shuiyisong
