rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-11-14 14:56:54 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore: merge narthex into feat/narthex-range-modifiers

Browse Source 
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

# Conflicts:
#	src/promql/src/native/display/tests.rs
This commit is contained in:
Dennis Zhuang
2026-10-22 23:42:27 +08:00
parent 3419266374 6c7e534b92
commit d273e1ef1f
36 changed files with 5502 additions and 385 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/formats/mito-samples.md
+650
View File
@@ -0,0 +1,650 @@
# Mito columnar samples format
**Status: implemented as an opt-in internal backend.**
This document specifies a standalone samples writer and reader. Query-consumer
integration, caches, and enabling samples by default are separate work.
The port is stacked on [#9841](https://github.com/GreptimeTeam/greptimedb/pull/9841),
at commit `6b44040c0ad96fb06d592c7bba5a1744e2274718`, and follows the presentation
and artifact conventions in [Mito columnar series](mito-series.md). Its sample
encodings and single-series blocks derive from the benchmark's
[columnar samples v2 specification][bench-spec] and [implementation][bench-code]
at revision `8650c3003a861c8cf2c9f5037bfc81fde6a3bdc4`.
The benchmark variant name does not determine the Mito wire version.
The principal format choices are nullable Float64 fields, preservation of source
rows and operation types, blocks bounded by rows and bytes without hourly splits,
and Mito framing with sample-specific time bounds. This is an independent format;
it does not read or write benchmark files.
## Artifact manifest
The catalog identifier is `columnar-samples`. Its file and manifest version is 1,
independent of the series format's version and both formats' codec identifiers.
Existing `parquet` and `columnar-series` identifiers keep their meaning.
One artifact is derived from one immutable source SST. It owns these objects:
```text
<region_id>/samples/<source_file_id>/<generation>/manifest.json
<region_id>/samples/<source_file_id>/<generation>/part-0.samples
<region_id>/samples/<source_file_id>/<generation>/part-1.samples
...
```
The source file ID and generation use existing `FileId` serialization. The
generation is unique for each build. A manifest contains:
| Field | Meaning |
| --- | --- |
| `format` | String `columnar-samples` |
| `version` | Integer 1 |
| `region_id` | Physical region ID, u64 |
| `source_file_id` | SST `FileId` matching `SamplesEntry.file_id` |
| `generation` | `FileId` matching the catalog artifact generation |
| `timestamp` | Object with `column_id`, `name`, and `unit` |
| `fields` | Ordered objects with `column_id`, `name`, `type`, and `nullable` |
| `files` | Ordered part descriptors, defined below |
The timestamp's ID is u32, its name is the region's time-index name, and its unit
is one of `second`, `millisecond`, `microsecond`, or `nanosecond`. Values are
signed i64 counts since the Unix epoch in that unit. There is no dataset-relative
origin, unit conversion, or timezone in the file. The timestamp is non-null.
The field list records the build-time region field schema in metadata order.
Require 1..=1,024 fields, each with `type = "float64"` and `nullable = true`.
A non-nullable source Float64 field is represented by this same nullable logical
type with no null values. Sparse logical-table fields may be entirely null.
Reject any other source field type; do not silently drop or cast it. Timestamp
and field IDs must be distinct, and all output names must be unique, including
the internal names below. Tags are not sample fields and are not stored here.
The stored logical Arrow schema is `__table_id: UInt32`, `__tsid: UInt64`, the
timestamp column with its recorded unit, `__sequence: UInt64`, `__op_type: UInt8`,
then nullable Float64 fields in manifest order. The first five columns are
non-null. No `__primary_key` or tag columns are exposed by the format reader.
Schema reconciliation with a later region schema belongs to a future consumer;
the reader always exposes the stored schema.
Each `files` entry contains `part_id: u32`, `file_size: u64`, `num_rows: u64`,
`num_blocks: u64`, `first`, `last`, `min_timestamp: i64`, and `max_timestamp: i64`.
`first` and `last` are inclusive `{ table_id: u32, tsid: u64 }` bounds. Time bounds
are the minimum and maximum over **all** blocks in the part, not just its first
and last rows. Every part is nonempty. Part IDs start at zero, increase by one,
and derive `part-<part_id>.samples` names with decimal IDs and no leading zeros.
No object path is stored in a part descriptor.
Parts follow input order. Adjacent identifier ranges satisfy
`previous.last <= next.first`; equality is allowed because a series can continue
in another part. The shared boundary series can also have equal timestamps on
both sides. These ranges are **not disjoint**, unlike the series-index format.
Readers consider every matching part rather than choosing a unique owner.
The manifest is written only after all parts close successfully. Existing samples
catalog publication commits the artifact. `ArtifactMetadata.objects` records the
manifest and every part as `{ path, file_size }`; the artifact's `file_size` is
their checked sum. Record sizes while writing, including the manifest's size.
An empty writer produces a manifest with an empty file list and no part files;
the existing builder may instead skip an SST for which it has no readable input.
On open, bound the manifest to 64 MiB before parsing. Validate its format, schema,
part order, and sizes; match region, source SST, and generation to the pinned handle.
Require the artifact object list to contain exactly the derived manifest and part
paths, without duplicates, extra objects, or inconsistent size totals. Read order
comes from the manifest, not the lifecycle object list.
## Part file
```text
┌──────────────────────────────────────────────┐
│ Header 16 bytes │
├──────────────────────────────────────────────┤
│ Independently encoded/compressed chunks │
│ Block 0: time, sequence, operation, fields │
│ Block 1: ... │
├──────────────────────────────────────────────┤ ← payload_end
│ Leaf pages (one is the root at depth 1) │
│ Optional internal root (depth 2) │
├──────────────────────────────────────────────┤
│ Trailer 112 bytes │
└──────────────────────────────────────────────┘
```
All fixed-width numbers are little-endian, without alignment padding. Byte ranges
are half-open; offsets are absolute from the start of the part. Parse each field
explicitly. Rust struct layout, enum order, and `usize` are not wire contracts.
Reserved fields must be zero, and decoded structures must have no trailing bytes.
### Header
| Field | Bytes | Value |
| --- | ---: | --- |
| Magic | 8 | ASCII `GTSAMPLE` |
| File version | 4 | 1 |
| Reserved | 4 | Zero |
### Column payloads and row semantics
Each block contains exactly one `(table_id, tsid)`. Identity is stored once in
its descriptor. Timestamp, sequence, operation type, and each present value field
are independent chunks; their references are in the metadata, not inline headers.
Emit chunks in that order, with value fields following manifest order.
The input is the flat `RecordBatch` stream from the existing per-SST samples
builder, after its `FlatCompatBatch` schema adaptation. Preserve every row
delivered by that builder. The writer neither deduplicates timestamps nor chooses
the highest sequence. It preserves delete markers, duplicate rows, and equal-key
ties, including conflicting values at the same sequence. This is not a statement
that an SST itself retains every historical write.
Require nondecreasing identifiers, nondecreasing timestamps within an identifier,
and nonincreasing sequence within equal timestamps, across batches and parts.
Equal ordering keys retain input order; operation type is not a sorting key.
Reject decreasing input rather than sorting or buffering a whole SST. Future
query adapters remain responsible for snapshot visibility, cross-SST merging,
deduplication, and delete processing.
`(table_id, tsid)` is the complete identity under #9841's current uniqueness
assumption. Equal TSIDs in different tables are distinct. Exact sparse primary
keys and tags are not retained, and this format does not promise collision-safe
identity. TODO: evaluate exact-key storage and collision handling with the series
format before using these sidecars as a general query source.
TODO: evaluate adding dedicated `job_id` and `instance_id` components to the
identifier for locality, following the VM native approach. The current
`(table_id, tsid)` pair is the version-1 identity, not a permanent restriction on
future formats. The extended identifier's component order, widths, derivation,
and relationship to TSID uniqueness require a separate design and benchmarks.
Coordinate that change with the series format: sample block identities,
part/page bounds, sorting, and selected-series requests must agree. Changing
the persisted identifier layout requires a new format version; do not repurpose
reserved bytes or silently reinterpret existing keys. Ordinary `job` and
`instance` tags do not become dedicated identifier components in this port.
Blocks split on an identity change or before the next row would exceed a packing
limit. There is no hourly split and no padding of short blocks. The default row
limit is 8,192, also the version-1 reader maximum; a writer may choose a smaller
positive limit without changing the version. With `F` manifest fields and `N`
rows, the decoded accounting size is:
```text
N * (8 timestamp + 8 sequence + 1 operation + 8 * F values)
+ F * ceil(N / 8) validity bytes
```
Require this to be at most 262,144 bytes. Count all manifest fields, including
omitted/all-null fields, and reserve validity bytes even for all-valid fields.
This is a bound on decoded column buffers, not total allocator usage or RSS.
The effective row cap is the largest integer satisfying both limits: 8,192 for
one field and 31 for 1,024 fields. Encoded buffers and metadata have separate
bounds below. Increasing the reader maximum requires an explicit compatibility
decision; it is not merely a writer option in this version.
Materializing the two identity columns adds at most `12 * N` bytes to an output
batch. Thus a full-schema output batch has at most `262144 + 12 * 8192 = 360448`
bytes of value/validity buffers under this accounting, plus Arrow and allocation
overhead. Temporary encoded, compact non-null, and selected-output buffers can
coexist; the block limit is not a promise that total reader memory is 256 KiB.
### Payload reference
Every stored column and metadata page uses the same 32-byte reference layout as
the series format:
| Field | Bytes | Meaning |
| --- | ---: | --- |
| Offset | 8 | Absolute location of stored bytes |
| Stored length | 4 | Bytes to fetch |
| Decoded length | 4 | Bytes after decompression, before column/page decoding |
| Checksum | 8 | XXH64 of stored bytes, seed zero |
| Encoding ID | 2 | Sample column or metadata encoding, according to context |
| Compression ID | 2 | Compression of the encoded bytes |
| Reserved | 4 | Zero |
Lengths are nonzero. Check `offset + stored_length` for u64 overflow before
checking the enclosing area. u32 payload lengths do not limit the part or its
offsets to 4 GiB. Verify the checksum before decompression; require exactly the
declared decoded length, with bounded output even if a codec frame claims more.
Checksums detect accidental corruption, not malicious modification.
### Metadata pages
A leaf describes blocks. An internal page references leaves. The root is either
the only leaf (depth 1) or an internal page (depth 2). Other depths are unsupported
in version 1; roll parts before a third level would be needed.
As in the series format, the entire page, including its kind and count, is encoded
and independently compressed. The reference in the parent or trailer supplies
its storage location, lengths, checksum, and codecs. There is no separate stored
page header. Decoded pages begin with `kind: u32` and `entry_count: u64`:
```text
Leaf (kind 0): Internal root (kind 1):
kind u32 kind u32
entry_count u64 entry_count u64
block descriptor 0 page reference 0 72 bytes
... ...
block descriptor N-1 page reference N-1 72 bytes
```
Pages are nonempty and limited to 65,536 decoded bytes, including the 12-byte
prefix. Descriptors never straddle pages. Unlike series label descriptors,
sample descriptors have no variable-size strings, and even the widest supported
descriptor fits in one page. No oversized-leaf exception is needed.
#### Block descriptor
| Field | Bytes | Meaning |
| --- | ---: | --- |
| Table ID | 4 | Constant for this block |
| TSID | 8 | Constant for this block |
| Minimum timestamp | 8 | Signed i64, first row's timestamp |
| Maximum timestamp | 8 | Signed i64, last row's timestamp |
| Block ordinal | 8 | Per-identifier ordinal across the artifact |
| Row count | 4 | Nonzero, subject to decoded-block limits |
| Column count | 4 | Number of following column descriptors |
| Column descriptors | 44 each | Defined below |
The fixed prefix is 44 bytes. Ordinals start at zero for each identity and
increase by one across block and part boundaries. Blocks sort lexicographically
by `(table_id, tsid, min_timestamp, ordinal)`. Adjacent blocks of the same series
satisfy `previous.max_timestamp <= next.min_timestamp`; equality is permitted.
Each column descriptor is:
| Field | Bytes | Meaning |
| --- | ---: | --- |
| Role | 4 | 1 timestamp, 2 sequence, 3 operation type, 4 value field |
| Column ID | 4 | Timestamp/field physical ID; zero for sequence/operation |
| Payload reference | 32 | Independently stored chunk |
| Null count | 4 | Number of null rows |
Role and column ID together identify a chunk. The role namespace avoids reserving
physical field IDs for internal columns. Time, sequence, and operation descriptors
are mandatory, appear first in that order, and have zero null counts. Value
descriptors follow manifest order with no repeated or unknown IDs. A value with
all rows null is omitted and synthesized as null by the reader; present value
chunks have `null_count < row_count`. No value statistics are stored in this port.
A block with `P` present fields has `C = 3 + P` chunks and a descriptor of
`44 + 44 * C` bytes. This is 220 bytes for one present field and 45,232 bytes for
1,024 present fields. The leaf prefix makes the latter 45,244 bytes. All chunks
are physically ordered and disjoint in the column area, across blocks as well
as within a block. There is no inline block header or block-wide compression.
#### Page reference and time bounds
Samples extend the series format's reference with two timestamp bounds:
| Field | Bytes | Meaning |
| --- | ---: | --- |
| First table ID; first TSID | 4 + 8 | Inclusive first identifier |
| Last table ID; last TSID | 4 + 8 | Inclusive last identifier |
| Minimum timestamp | 8 | Minimum over all descendant blocks |
| Maximum timestamp | 8 | Maximum over all descendant blocks |
| Payload reference | 32 | Location and codecs of the child page |
The reference is 72 bytes. Compare identifiers as integer pairs, not as serialized
little-endian bytes. Time bounds use the manifest unit. The time range is an
envelope over all identifiers in the page; it need not describe the first or last
identifier alone. This can admit extra reads, but cannot exclude matching rows.
Every part/page identifier interval requires `first <= last`, and all part/page/
block time intervals require `min_timestamp <= max_timestamp`, even when pruning
would skip their payloads.
Child pages follow block order. Their identifier bounds may touch at a shared
series, just like part bounds. Do not require strict disjointness or use an
exclusive next-child lower bound. Validate each loaded page's derived identifier
and time bounds against its reference. At depth 2 all children must be leaves;
reject an internal child. Child byte ranges are ordered, disjoint, within the
metadata area, and precede the parent. The root ends immediately before the trailer.
An internal root has `12 + 72 * N` decoded bytes, so it holds at most
`floor((65536 - 12) / 72) = 910` leaf references. The writer must account for the
next descriptor's leaf placement before writing its block. If the descriptor
would require a 911th leaf, finish this part and start the block in a new part.
#### Navigation and equal-timestamp example
For a requested qualified identity and optional half-open interval `[start, end)`,
select every part/page whose identifier interval contains the identity and whose
time envelope overlaps the request:
```text
(start is absent || max_timestamp >= start)
&& (end is absent || min_timestamp < end)
```
For multiple selected identities, intersect the ordered ID set with the inclusive
identifier range. A table restriction also intersects this selection. Visit each
selected page once per request. Apply the same overlap test to leaf blocks, then
filter decoded timestamps exactly. Metadata bounds are conservative candidates.
No predecessor-only or unique-child lookup is used: a shared identity or timestamp
can require several adjacent children, and even several parts.
```text
Part 0, depth 2:
Root R
├─ Leaf A: IDs [(7,42),(7,42)], times [0,100]
│ block ordinal 0: (7,42), times [0,100]
└─ Leaf B: IDs [(7,42),(7,43)], times [0,200]
block ordinal 1: (7,42), times [100,200]
block ordinal 0: (7,43), times [0,50]
Part 1:
Leaf/root C: IDs [(7,43),(7,43)], times [50,80]
block ordinal 1: (7,43), times [50,80]
```
For `(7,42)` and `[100,101)`, visit both A and B, keeping all rows at timestamp
100 from blocks 0 and 1. Do not decode `(7,43)` payloads. For `(7,43)` and
`[50,51)`, visit B and part 1's C: both may contain rows at 50. A `[51,60)` request
for `(7,43)` can skip B's `(7,43)` block even though B's time envelope overlaps.
For `(7,42)` and `[90,95)`, A is still visited although its block starts before
the requested interval. Global time envelopes must not be computed from just
the first and last blocks: B's range is `[0,200]`, not `[100,50]`.
This differs from #9841, which has no sample timestamps or time navigation.
### Trailer
| Field | Bytes | Meaning |
| --- | ---: | --- |
| Root page reference | 72 | Inclusive ID/time bounds and payload reference |
| Index depth | 4 | 1 or 2 |
| File version | 4 | 1 |
| File length | 8 | Includes header and trailer |
| `payload_end` | 8 | End of column area, start of metadata area |
| Magic | 8 | ASCII `GTSAMPLE` |
| Trailer checksum | 8 | XXH64 of the preceding 104 trailer bytes, seed zero |
The trailer is 112 bytes. Locate it using the recorded object size without a stat
request. Validate the checksum, magic, version, length, depth, area boundaries,
root bounds against the manifest part descriptor, and root end position before
traversal. Column references must lie in `[16, payload_end)`; page references lie
in `[payload_end, file_length - 112)`. The header and trailer versions must agree.
## Encodings and compression
Sample column encoding IDs below are distinct from series column encoding IDs,
even where their numeric values coincide. Metadata and compression have separate
namespaces as well. Never reuse an assigned ID with a different meaning.
| Namespace | ID | Meaning |
| --- | ---: | --- |
| Column encoding | 1 | Timestamp constant step |
| Column encoding | 2 | Timestamp delta-of-delta |
| Column encoding | 3 | Constant sequence |
| Column encoding | 4 | Sequence delta |
| Column encoding | 5 | Nullable Float64 XOR |
| Column encoding | 6 | Constant operation type |
| Column encoding | 7 | Raw operation types |
| Metadata encoding | 1 | Leaf/internal page layout above |
| Compression | 0 | Raw encoded bytes |
| Compression | 1 | Zstd frame |
Encode first, then compress each chunk/page independently. The default writer
tries Zstd level 3 and chooses it only if strictly smaller than the raw encoded
bytes, following the sample benchmark. Compression 0 requires equal stored and
decoded lengths. Compression level is a writer choice; decoding information is
in the frame. Codec selection adds no public configuration.
Unsigned varints use base-128/LEB128: seven data bits per byte and a high
continuation bit. Writers emit the shortest representation. Readers reject
truncation, more than ten bytes, a tenth byte greater than 1, and trailing bytes
after the expected values. ZigZag maps signed i64 `x` to
`((x as u64) << 1) ^ ((x >> 63) as u64)` and reverses that mapping on decode.
All wrapping arithmetic below is modulo 2^64, not signed overflowing arithmetic.
### Timestamp encodings 1 and 2
Encoding 1 contains `first: i64` and `step: i64`, exactly 16 bytes. Use it when
every adjacent difference is the same nonnegative, representable i64. A singleton
uses step zero. Reconstruct by checked addition; reject overflow. Zero step is
valid for repeated timestamps.
Encoding 2 contains the first i64, followed for two or more rows by the first
delta as a ZigZag varint, then ZigZag varints of differences between deltas.
Compute deltas and delta differences with wrapping arithmetic; reconstruct in
the same way. This permits differences across the full signed timestamp range.
After either decoding, require nondecreasing signed timestamps and exact agreement
of first/last values with the descriptor. Consume exactly the recorded row count.
Use encoding 1 when applicable, otherwise encoding 2.
### Sequence encodings 3 and 4
Encoding 3 contains one u64, repeated for all rows. Encoding 4 contains the first
u64 followed by each wrapping u64 difference from its predecessor, reinterpreted
as i64 and written as a ZigZag varint. Decode with wrapping addition. Use 3 if
all sequences are equal, otherwise 4. Preserve all u64 bit patterns. Sequence
need not be monotonic across different timestamps; equal-timestamp runs retain
the input's nonincreasing order.
### Nullable Float64 encoding 5
```text
validity_kind <u8> # 0 = all valid, 1 = bitmap
validity <ceil(row_count/8) bytes> # only for kind 1
first_non_null_bits <u64>
xor_deltas <one uvarint per remaining non-null value>
```
Bitmap bit `i % 8` in byte `i / 8` is one for a present value; unused high bits of
the final byte are zero. Kind 0 has no bitmap and requires zero null count. Kind
1 requires `0 < null_count < row_count` and an exactly matching count of unset
in-range bits. All-null columns have no descriptor or payload, so every stored
Float64 chunk contains at least one non-null value.
Process non-null values in row order. Write the first IEEE-754 bit pattern as
u64, then varints of XOR with the previous **non-null** bit pattern. Nulls do not
reset that previous value. Decode exactly `row_count - null_count` values and
scatter according to validity. Preserve all bits, including signed zero, NaN
payloads, infinities, and subnormals. Null and NaN are distinct. The unspecified
value-buffer bits behind nulls are not part of the logical preservation contract.
### Operation encodings 6 and 7
Operation values are u8: 0 delete, 1 put, matching Mito's current `OpType` values.
Unknown operation values fail decoding. Encoding 6 stores one byte repeated for
the whole block; encoding 7 stores exactly one byte per row. Use 6 when all values
are equal, otherwise 7. Neither encoding filters delete rows or discards their
field values.
### Limits and evolution
An individual chunk has at most 1 MiB of encoded bytes after decompression and
1 MiB plus 64 KiB of stored bytes. Pages have at most 65,536 decoded and 131,072
stored bytes. Bound sizes before allocating, fetching, or decompressing. Validate
counts against enclosing bytes before allocating descriptor vectors; row counts
must also satisfy the decoded-block formula. Null output materialization remains
subject to the same block bound even when chunks are omitted.
Unknown file versions, depths, reserved fields, roles, or metadata codecs fail
open/read explicitly. Column role must agree with its encoding: time 1/2,
sequence 3/4, Float64 5, operation 6/7. An unknown column encoding or compression
can be skipped if its chunk is not needed, but attempting to decode it returns
an unsupported-codec error. Known encodings assigned to a different role are
invalid metadata, even for an unprojected column. Unknown fields or malformed
references must not be interpreted as omitted all-null columns.
New column encodings or compression methods receive new IDs. Structural changes,
changed field meanings, or incompatible limits require a new version. Deeper
metadata trees require an explicit reader-support and rollout decision; an
unchanged layout alone does not make them readable by version-1 implementations.
## Writer, OpenDAL I/O, and lifecycle
Implement behind `SamplesFormat::writer` and `FormatWriter`, using the existing
`object_store::ObjectStore` OpenDAL operator. Extend the internal writer factory
with an explicit source SST `FileId` so the manifest does not infer identity by
parsing an opaque prefix. Check that the supplied prefix agrees with metadata,
source ID, and generation. The builder supplies these values from its pinned SST
and captured region version; fake backends follow the same internal interface.
Validate supported sparse primary-key metadata and extract `(table_id, tsid)`
with the same conventions as the series port. Accept the builder's binary or
dictionary primary-key array representation. Validate column types and mandatory
non-null values before writing; preserve ordering state across `write` calls.
Retain only the active block and bounded per-part descriptors, not all SST rows.
No external sort, benchmark manifest/schema file, or filesystem scratch spool is
introduced.
Write chunks as each block completes. Finish a part at the next block boundary
once its stored position reaches 512 MiB or its accumulated serialized block
descriptors reach 8 MiB. Both are soft packing limits; one bounded block may
overshoot. Root capacity is a separate hard limit checked before emitting that
block. Write leaves, the internal root if needed, and the trailer, then close the
part before opening its successor. Bound the manifest and lifecycle object-list
growth as parts complete; fail rather than constructing a manifest over 64 MiB.
Use recorded sizes for reads; do not issue object stats just to locate trailers.
Fetch metadata and selected column ranges only. Reads are sequential in this
initial port, without speculative adjacent-field fetches or prefetch. Keep
fetch/validate/decode entrypoints for metadata and columns separate so a later
cache integration can use immutable generation, part, byte range, checksum,
codec IDs, and representation without changing the format.
Writers own all partial outputs until `finish` succeeds. Write/close errors,
explicit abort, and cancellation clean up every created object, including an
unfinished manifest. Use the existing cleanup/runtime pattern from #9841;
cleanup errors must not hide the original error. A process crash can leave orphan
objects, but never publishes an unfinished generation as valid. Reconciliation
and existing orphan handling remain responsible for recovery.
After finish, ownership transfers to the build/catalog lifecycle. Failed catalog
publication retires unpublished outputs and preserves the prior catalog. Each
reader and returned stream holds a `SamplesFileHandle`; superseded objects are
purged only after those handles are released. Sample sidecars remain rebuildable,
and authoritative data stays in Parquet SSTs.
## Reader API and execution
Add a sample-specific reader factory to `SamplesFormat`, accepting the store and
a pinned `SamplesFileHandle`. Provide access to the handle's region/source identity
for manifest validation. Return a `SamplesFormatReader` exposing `schema()` and
`read(SamplesReadRequest)` as an independent stream of projected Arrow batches.
Do not reuse the series-only read request: it has no time semantics.
The internal request has these fields:
| Field | Meaning |
| --- | --- |
| `projection: Vec<String>` | Ordered names from the stored logical schema |
| `table_id: Option<u32>` | Optional table restriction |
| `ids: Option<BTreeSet<MetricSeriesId>>` | Optional qualified identity set |
| `start: Option<i64>` | Inclusive timestamp in the stored unit |
| `end: Option<i64>` | Exclusive timestamp in the stored unit |
`ids = None` means unrestricted; an explicitly empty set means zero rows.
Intersect `ids` with `table_id` when both are supplied. Validate unknown or
duplicate projection names before part I/O; preserve requested output order.
An empty projection preserves selected row counts. Equal time endpoints produce
an empty stream, and a reversed interval is an error. Unbounded endpoints avoid
sentinel arithmetic and permit selecting `i64::MAX` with no end bound. Callers
perform any unit conversion explicitly; the format API does not guess units.
1. The factory opens the bounded manifest, validates handle identity and recorded
sizes, and exposes the stored schema. Each `read` validates its request even
if the artifact or selection is empty. An empty selection needs no part reads.
2. Prune parts using inclusive ID ranges and time envelopes. Read and validate
selected headers/trailers, then traverse only candidate pages in physical
order. At depth 2, retain the root references and one active leaf.
3. Select blocks by exact identity and overlapping time bounds. Decode timestamp
chunks for a constrained time request, validate their bounds, and select rows.
With no time restriction, an unprojected timestamp chunk need not be fetched.
4. Fetch the remaining projected chunks only for blocks with surviving rows,
reusing a timestamp chunk already decoded for selection. Decode each chunk
once per block, synthesize identity constants and omitted null fields, and
emit selected rows in input order. One output batch per selected block is the
initial packing choice, with the additional identity-column accounting above.
Sequence and operation chunks are mandatory on disk but are read only if
projected. The format reader does not silently apply visibility or delete rules.
A future query adapter must request them when required for merging. There are
no tag/value predicates in this initial request, no selector-result cache, and
no cross-read page/data cache. All reads have independent traversal state and
can outlive the factory. Validation is incremental over visited metadata and
decoded chunks, not an eager whole-artifact integrity scan; unselected payload
corruption is not guaranteed to be discovered by a selective query.
Check block ordinals and row ordering across adjacent blocks when both are read;
do not require the first selected block to have ordinal zero after pruning.
A complete traversal also verifies the manifest's row and block totals. These
totals are diagnostic summaries, not allocation sizes or a replacement for block
row counts when producing output.
## Compatibility, validation, and deferred work
The backend is opt-in through internal format selection, like #9841. Existing catalogs, Parquet
artifacts, and series artifacts retain their meaning. An older binary is not
promised to understand the new catalog identifier or sample bytes; do not claim
downgrade compatibility merely because the samples are rebuildable. The new
backend must not become necessary for reading authoritative SSTs. Query-consumer
integration and a public enablement/rollout mechanism are separate work.
Implementation acceptance includes:
- Codec tests and round trips for every supported timestamp unit, repeated and
negative timestamps, signed extremes, full-range sequences, both operations,
null/NaN distinction, Float64 bit patterns, and nulls between XOR values.
- Empty input, singleton blocks, all-null/missing fields, mixed tables and sparse
schemas, identical TSIDs in different tables, cross-batch continuation,
duplicate/conflicting rows, and decreasing-input rejection.
- Row and decoded-byte caps with narrow/wide schemas; smaller row limits; blocks
spanning hours; forced payload/metadata rollover; depths 1/2 and root-capacity
rollover without creating depth 3. Reject depths 0/3 and internal children.
- Projection order and empty projection, all/empty/absent ID sets, table/ID
intersection, exact time boundaries, unbounded endpoints, and equal timestamps
spanning multiple leaves and parts. Verify selected physical ranges: no unused
value chunks, duplicate page reads, or hidden stat requests.
- Truncation, corrupt checksums, invalid area bounds, u64 offset overflow,
references beyond 4 GiB, wrong lengths/counts/roles/ordinals, inconsistent
time bounds, malformed validity/varints, unsupported codecs, and bounded
decompression. Test malformed metadata without allocating from untrusted counts.
- Manifest/schema/handle identity mismatches, wrong or duplicate object paths,
size accounting, reopen and reconciliation, OpenDAL write/close failures,
abort/cancellation, failed catalog publication, and retirement with pinned
streams on memory and filesystem stores.
- A persisted-format compatibility case under `tests/compatibility/`, following
its [README](../../tests/compatibility/README.md) and
[agent guide](../../tests/compatibility/AGENTS.md), covering the approved version-1
bytes and legacy catalog behavior. Do not substitute a SQL query test that
never enables this internal backend or claim cross-binary coverage from a
same-build round trip.
- `cargo nextest run -p mito2`, crate Clippy, formatting, and license checks for
new sources. Any broader workspace validation and its limitations must be
reported when the implementation PR is prepared.
The benchmark comparison changes materially: nullable fields and operation types
add bytes; native timestamp units and preserved duplicate rows change codec
inputs; 64-bit checksums, Mito manifests/references, and overlapping page bounds
change framing; bounded two-level trees replace disk-spooled arbitrary-depth
indexes; row/byte packing replaces the benchmark's hourly default. OpenDAL and
the absence of caches also change performance. Measure the port independently;
no performance equivalence is asserted.
The benchmark follow-ups from [PR #9887](https://github.com/GreptimeTeam/greptimedb/pull/9887)
include:
- High-churn series with only a few rows per SST. Report stored bytes per row
and separate metadata from column payloads across different rows-per-series
distributions. Each identity change starts a new block, whose descriptor is
220 bytes before metadata compression with one present field, in addition to
the mandatory timestamp, sequence, and operation chunks and the value chunk.
- Integer-valued Float64 counters. Compare the current XOR encoding with an
exact `first`/`step` progression encoding like the prototype in
[PR #9556](https://github.com/GreptimeTeam/greptimedb/pull/9556), measuring encoded
and compressed sizes as well as encode/decode throughput. Include steady
counters, resets, irregular increments, and nulls; any candidate must preserve
Float64 bits exactly and fall back for values it cannot represent losslessly.
These measurements are deferred and do not change the version-1 wire contract.
Additional field types, value statistics/predicates, exact primary keys,
dedicated job/instance identifier components, label reconstruction, cache
integration, deeper trees, and query integration are deferred. The byte assignments
above define the version-1 contract. The
[frozen sample fixture](../../tests/compatibility/wire/mito_columnar_samples/README.md)
checks those bytes independently of writer/reader round trips.
[bench-spec]: https://github.com/GreptimeTeam/series-format-bench/blob/8650c3003a861c8cf2c9f5037bfc81fde6a3bdc4/docs/formats/columnar-samples-v2.md
[bench-code]: https://github.com/GreptimeTeam/series-format-bench/blob/8650c3003a861c8cf2c9f5037bfc81fde6a3bdc4/src/sample_columnar.rs
src/mito2/AGENTS.md
+5
View File
@@ -69,6 +69,11 @@ filtered `RecordBatch` stream.
- **Series format PoC**: samples are rebuildable local SST sidecars. Optional writer interfaces (`format.rs`) integrate with reconciliation, catalog publication, and deferred purge. Defaults retain the original Parquet writer and disable samples. The opt-in columnar backend (`series_index/columnar.rs`) implements streaming OpenDAL writes and pinned format reads; its wire contract is in `docs/formats/mito-series.md`. Artifact objects record paths and sizes, including the manifest, so reads need no stat. Codecs currently run inline pending scheduling benchmarks. Prepared sources and query-consumer integration are deferred; ordinary scans only use Parquet index artifacts.
- **Series-index coverage** (`series_index/catalog.rs`): `SeriesIndexEntry` stores
compaction-window width and SST summaries keyed by aligned start in both catalogs and Parquet footers.
- **Columnar samples**: `series_index/samples.rs` implements opt-in per-SST sidecars,
including nullable Float64 fields and preserved sequence/delete rows. Its wire
contract is `docs/formats/mito-samples.md`; time/ID bounds may overlap across
leaves and parts. Both columnar backends share cancellation-safe output ownership
in `series_index/artifact_writer.rs`. Sample reads do not apply query visibility.
- **Request types** (`request.rs`): usually tied to proto definitions consumed by `datanode`.
- **WAL/memtable encoding** (`wal/`, `memtable/`): breaks replay if changed incompatibly.
src/mito2/src/series_index.rs
+6
View File
@@ -17,6 +17,7 @@
//! Under development. Index files are currently stored on the local filesystem.
pub(crate) mod artifact;
mod artifact_writer;
mod bucket;
mod builder;
mod catalog;
@@ -30,6 +31,11 @@ pub(crate) mod format;
mod format_tests;
mod maintenance;
mod purger;
#[allow(
dead_code,
reason = "Samples are an opt-in backend; query integration is deferred."
)]
pub(crate) mod samples;
mod searcher;
mod task;
#[cfg(test)]
src/mito2/src/series_index/artifact.rs
+6
View File
@@ -30,6 +30,8 @@ pub(crate) enum ArtifactFormat {
Parquet,
#[serde(rename = "columnar-series")]
ColumnarSeries,
#[serde(rename = "columnar-samples")]
ColumnarSamples,
#[cfg(test)]
#[serde(rename = "fake-samples")]
FakeSamples,
@@ -83,6 +85,10 @@ impl SamplesFileHandle {
}))
}
pub(crate) fn file_id(&self) -> RegionFileId {
self.0.file_id
}
pub(crate) fn entry(&self) -> &SamplesEntry {
&self.0.entry
}
src/mito2/src/series_index/artifact_writer.rs
+144
View File
@@ -0,0 +1,144 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Owned outputs shared by immutable columnar artifacts.
use bytes::Bytes;
use object_store::{ObjectStore, Writer};
use snafu::{IntoError, ResultExt};
use crate::error::{JoinSnafu, OpenDalSnafu, Result};
use crate::series_index::columnar::{invalid, require};
use crate::sst::{DEFAULT_WRITE_BUFFER_SIZE, DEFAULT_WRITE_CONCURRENCY};
/// Drop owns the live OpenDAL writer, including its private atomic-write temporary file.
pub(super) struct LiveFile {
writer: Option<Writer>,
store: ObjectStore,
path: String,
}
impl Drop for LiveFile {
fn drop(&mut self) {
if let Some(mut writer) = self.writer.take() {
let store = self.store.clone();
let path = self.path.clone();
common_runtime::spawn_compact(async move {
if let Err(error) = writer.abort().await {
common_telemetry::warn!(error; "Aborting artifact output {path}");
}
if let Err(error) = store.delete(&path).await {
common_telemetry::warn!(error; "Deleting artifact output {path}");
}
});
}
}
}
pub(super) struct Outputs {
pub(super) store: ObjectStore,
pub(super) active: Option<LiveFile>,
pub(super) paths: Vec<String>,
pub(super) committed: bool,
}
impl Outputs {
pub(super) async fn open(&mut self, path: String) -> Result<()> {
require(self.active.is_none(), "artifact output already open")?;
self.paths.push(path.clone());
let store = self.store.clone();
// The task owns opening: cancellation drops the returned LiveFile and aborts it.
self.active = Some(
common_runtime::spawn_compact(async move {
let writer = store
.writer_with(&path)
.chunk(DEFAULT_WRITE_BUFFER_SIZE.as_bytes() as usize)
.concurrent(DEFAULT_WRITE_CONCURRENCY)
.await
.context(OpenDalSnafu)?;
Ok::<_, crate::error::Error>(LiveFile {
writer: Some(writer),
store,
path,
})
})
.await
.context(JoinSnafu)??,
);
Ok(())
}
pub(super) async fn write(&mut self, bytes: Vec<u8>) -> Result<()> {
self.active
.as_mut()
.and_then(|f| f.writer.as_mut())
.ok_or_else(|| invalid("artifact output is not open"))?
.write(Bytes::from(bytes))
.await
.context(OpenDalSnafu)?;
Ok(())
}
pub(super) async fn close(&mut self) -> Result<()> {
let file = self
.active
.as_mut()
.ok_or_else(|| invalid("artifact output is not open"))?;
file.writer
.as_mut()
.ok_or_else(|| invalid("artifact output is closed"))?
.close()
.await
.context(OpenDalSnafu)?;
file.writer = None;
self.active = None;
Ok(())
}
pub(super) async fn cleanup(&mut self) -> Result<()> {
let mut first_error = None;
if let Some(file) = self.active.as_mut()
&& let Some(writer) = file.writer.as_mut()
{
match writer.abort().await {
Ok(()) => file.writer = None,
Err(error) => first_error = Some(OpenDalSnafu.into_error(error)),
}
}
self.active = None;
for path in &self.paths {
if let Err(error) = self.store.delete(path).await {
first_error.get_or_insert_with(|| OpenDalSnafu.into_error(error));
}
}
if let Some(error) = first_error {
Err(error)
} else {
self.committed = true;
Ok(())
}
}
}
impl Drop for Outputs {
fn drop(&mut self) {
if !self.committed {
let mut owned = Self {
store: self.store.clone(),
active: self.active.take(),
paths: std::mem::take(&mut self.paths),
committed: true,
};
common_runtime::spawn_compact(async move {
if let Err(error) = owned.cleanup().await {
common_telemetry::warn!(error; "Cleaning unpublished format artifact");
}
});
}
}
}
src/mito2/src/series_index/builder.rs
+7 -1
View File
@@ -277,7 +277,13 @@ pub(crate) async fn build_samples(
file_id.file_id()
);
let mut writer = backend
.writer(version.metadata.clone(), store.clone(), &prefix, generation)
.writer(
version.metadata.clone(),
store.clone(),
&prefix,
file_id.file_id(),
generation,
)
.await?;
let result: Result<()> = async {
while let Some((row_group, selection)) = selection.pop_first() {
src/mito2/src/series_index/catalog.rs
+1
View File
@@ -314,6 +314,7 @@ mod tests {
for (format, name) in [
(ArtifactFormat::Parquet, "parquet"),
(ArtifactFormat::ColumnarSeries, "columnar-series"),
(ArtifactFormat::ColumnarSamples, "columnar-samples"),
(ArtifactFormat::FakeSamples, "fake-samples"),
(ArtifactFormat::FakeIndex, "fake-index"),
] {
src/mito2/src/series_index/columnar.rs
+1 -1
View File
@@ -14,7 +14,7 @@
//! Columnar series format. See `docs/formats/mito-series.md` for the wire contract.
mod codec;
pub(super) mod codec;
mod metadata;
mod reader;
#[cfg(test)]
src/mito2/src/series_index/columnar/writer.rs
+4 -126
View File
@@ -15,17 +15,17 @@
//! Streaming aggregation and owned OpenDAL outputs.
use async_trait::async_trait;
use bytes::Bytes;
use datatypes::arrow::array::{Array, BinaryArray, DictionaryArray};
use datatypes::arrow::datatypes::UInt32Type;
use datatypes::arrow::record_batch::RecordBatch;
use mito_codec::row_converter::SparseOffsetsCache;
use object_store::{ObjectStore, Writer};
use snafu::{IntoError, ResultExt};
use object_store::ObjectStore;
use snafu::ResultExt;
use store_api::metadata::RegionMetadataRef;
use crate::error::{JoinSnafu, OpenDalSnafu, Result, SerdeJsonSnafu};
use crate::error::{Result, SerdeJsonSnafu};
use crate::series_index::artifact::{ArtifactFormat, ArtifactMetadata, ArtifactObject};
use crate::series_index::artifact_writer::Outputs;
use crate::series_index::catalog::SeriesIndexEntry;
use crate::series_index::columnar::codec::{self, encode_dictionary, encode_integers};
use crate::series_index::columnar::metadata::{
@@ -37,132 +37,10 @@ use crate::series_index::columnar::{
use crate::series_index::format::FormatWriter;
use crate::series_index::writer::{decode_series_identity, tag_columns, validate_metadata};
use crate::series_index::{MetricSeriesId, ROW_COUNT_COLUMN, TSID_COLUMN};
use crate::sst::{DEFAULT_WRITE_BUFFER_SIZE, DEFAULT_WRITE_CONCURRENCY};
/// Maximum owned row bytes per block; a single larger row is flushed on its own.
const BLOCK_BYTE_BUDGET: usize = 8 * 1024 * 1024;
/// Drop owns the live OpenDAL writer, including its private atomic-write temporary file.
struct LiveFile {
writer: Option<Writer>,
store: ObjectStore,
path: String,
}
impl Drop for LiveFile {
fn drop(&mut self) {
if let Some(mut writer) = self.writer.take() {
let store = self.store.clone();
let path = self.path.clone();
common_runtime::spawn_compact(async move {
if let Err(error) = writer.abort().await {
common_telemetry::warn!(error; "Aborting series output {path}");
}
if let Err(error) = store.delete(&path).await {
common_telemetry::warn!(error; "Deleting series output {path}");
}
});
}
}
}
struct Outputs {
store: ObjectStore,
active: Option<LiveFile>,
paths: Vec<String>,
committed: bool,
}
impl Outputs {
async fn open(&mut self, path: String) -> Result<()> {
require(self.active.is_none(), "series output already open")?;
self.paths.push(path.clone());
let store = self.store.clone();
// The task owns opening: cancellation drops the returned LiveFile and aborts it.
self.active = Some(
common_runtime::spawn_compact(async move {
let writer = store
.writer_with(&path)
.chunk(DEFAULT_WRITE_BUFFER_SIZE.as_bytes() as usize)
.concurrent(DEFAULT_WRITE_CONCURRENCY)
.await
.context(OpenDalSnafu)?;
Ok::<_, crate::error::Error>(LiveFile {
writer: Some(writer),
store,
path,
})
})
.await
.context(JoinSnafu)??,
);
Ok(())
}
async fn write(&mut self, bytes: Vec<u8>) -> Result<()> {
self.active
.as_mut()
.and_then(|f| f.writer.as_mut())
.ok_or_else(|| invalid("series output is not open"))?
.write(Bytes::from(bytes))
.await
.context(OpenDalSnafu)?;
Ok(())
}
async fn close(&mut self) -> Result<()> {
let file = self
.active
.as_mut()
.ok_or_else(|| invalid("series output is not open"))?;
file.writer
.as_mut()
.ok_or_else(|| invalid("series output is closed"))?
.close()
.await
.context(OpenDalSnafu)?;
file.writer = None;
self.active = None;
Ok(())
}
async fn cleanup(&mut self) -> Result<()> {
let mut first_error = None;
if let Some(file) = self.active.as_mut()
&& let Some(writer) = file.writer.as_mut()
{
match writer.abort().await {
Ok(()) => file.writer = None,
Err(error) => first_error = Some(OpenDalSnafu.into_error(error)),
}
}
self.active = None;
for path in &self.paths {
if let Err(error) = self.store.delete(path).await {
first_error.get_or_insert_with(|| OpenDalSnafu.into_error(error));
}
}
if let Some(error) = first_error {
Err(error)
} else {
self.committed = true;
Ok(())
}
}
}
impl Drop for Outputs {
fn drop(&mut self) {
if !self.committed {
let mut owned = Self {
store: self.store.clone(),
active: self.active.take(),
paths: std::mem::take(&mut self.paths),
committed: true,
};
common_runtime::spawn_compact(async move {
if let Err(error) = owned.cleanup().await {
common_telemetry::warn!(error; "Cleaning unpublished series artifact");
}
});
}
}
}
#[derive(Debug)]
struct Row {
id: MetricSeriesId,
src/mito2/src/series_index/format.rs
+31 -2
View File
@@ -28,7 +28,7 @@ use store_api::metadata::RegionMetadataRef;
use store_api::storage::FileId;
use crate::error::Result;
use crate::series_index::artifact::{ArtifactFormat, ArtifactMetadata};
use crate::series_index::artifact::{ArtifactFormat, ArtifactMetadata, SamplesFileHandle};
use crate::series_index::catalog::SeriesIndexEntry;
use crate::series_index::{MetricSeriesId, SeriesIndexFileHandle};
@@ -87,16 +87,45 @@ pub(crate) trait SeriesIndexFormat: Debug + Send + Sync {
) -> Result<Box<dyn FormatWriter>>;
}
/// Builds samples artifacts; read interfaces are deferred.
/// Selection in the stored timestamp unit. No visibility filtering is implicit.
#[allow(dead_code, reason = "Query adapter is deferred.")]
#[derive(Debug, Clone, Default)]
pub(crate) struct SamplesReadRequest {
pub(crate) projection: Vec<String>,
pub(crate) table_id: Option<u32>,
pub(crate) ids: Option<BTreeSet<MetricSeriesId>>,
pub(crate) start: Option<i64>,
pub(crate) end: Option<i64>,
}
#[allow(dead_code, reason = "Query adapter is deferred.")]
pub(crate) trait SamplesFormatReader: Send + Sync {
fn schema(&self) -> SchemaRef;
fn read(&self, request: SamplesReadRequest) -> Result<BoxStream<'static, Result<RecordBatch>>>;
}
/// Builds and opens immutable per-SST samples artifacts.
#[async_trait]
pub(crate) trait SamplesFormat: Debug + Send + Sync {
fn format(&self) -> ArtifactFormat;
#[allow(dead_code, reason = "Query adapter is deferred.")]
async fn reader(
&self,
_store: ObjectStore,
_handle: SamplesFileHandle,
) -> Result<Box<dyn SamplesFormatReader>> {
crate::error::UnsupportedOperationSnafu {
err_msg: "This samples format has no reader",
}
.fail()
}
/// The caller supplies a unique prefix. Finish reports all objects owned below it.
async fn writer(
&self,
metadata: RegionMetadataRef,
store: ObjectStore,
prefix: &str,
source_file_id: FileId,
generation: FileId,
) -> Result<Box<dyn FormatWriter>>;
}
src/mito2/src/series_index/format_tests.rs
+1
View File
@@ -84,6 +84,7 @@ impl SamplesFormat for FakeSamples {
_metadata: RegionMetadataRef,
store: ObjectStore,
prefix: &str,
_source_file_id: FileId,
generation: FileId,
) -> Result<Box<dyn FormatWriter>> {
let objects = vec![format!("{prefix}data"), format!("{prefix}directory")];
src/mito2/src/series_index/samples.rs
+222
View File
@@ -0,0 +1,222 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Columnar samples sidecars. Wire contract: `docs/formats/mito-samples.md`.
mod codec;
mod metadata;
mod reader;
#[cfg(test)]
mod tests;
mod writer;
use std::collections::HashSet;
use std::sync::Arc;
use async_trait::async_trait;
use datatypes::arrow::datatypes::{DataType, Field, Schema, SchemaRef, TimeUnit};
use object_store::ObjectStore;
use serde::{Deserialize, Serialize};
use store_api::metadata::RegionMetadataRef;
use store_api::storage::FileId;
use crate::error::Result;
use crate::series_index::artifact::{ArtifactFormat, SamplesFileHandle};
use crate::series_index::columnar::{invalid, require, unsupported};
use crate::series_index::format::{FormatWriter, SamplesFormat, SamplesFormatReader};
use crate::series_index::{MetricSeriesId, TABLE_ID_COLUMN, TSID_COLUMN};
const VERSION: u32 = 1;
const MANIFEST_LIMIT: usize = 64 * 1024 * 1024;
const MAX_ROWS: usize = 8192;
const MAX_FIELDS: usize = 1024;
const BLOCK_BYTES: usize = 256 * 1024;
const SEQUENCE: &str = "__sequence";
const OPERATION: &str = "__op_type";
/// Internal packing knobs; the backend is disabled unless selected explicitly.
#[derive(Debug, Clone)]
pub(crate) struct ColumnarSamplesFormat {
pub(crate) block_rows: usize,
pub(crate) part_bytes: u64,
pub(crate) metadata_bytes: usize,
pub(crate) page_bytes: usize,
pub(crate) root_capacity: usize,
}
impl Default for ColumnarSamplesFormat {
fn default() -> Self {
Self {
block_rows: MAX_ROWS,
part_bytes: 512 * 1024 * 1024,
metadata_bytes: 8 * 1024 * 1024,
page_bytes: metadata::PAGE_LIMIT,
root_capacity: metadata::ROOT_CAPACITY,
}
}
}
#[async_trait]
impl SamplesFormat for ColumnarSamplesFormat {
fn format(&self) -> ArtifactFormat {
ArtifactFormat::ColumnarSamples
}
async fn writer(
&self,
metadata: RegionMetadataRef,
store: ObjectStore,
prefix: &str,
source_file_id: FileId,
generation: FileId,
) -> Result<Box<dyn FormatWriter>> {
Ok(Box::new(writer::SamplesWriter::new(
self.clone(),
metadata,
store,
prefix,
source_file_id,
generation,
)?))
}
async fn reader(
&self,
store: ObjectStore,
handle: SamplesFileHandle,
) -> Result<Box<dyn SamplesFormatReader>> {
Ok(Box::new(reader::SamplesReader::open(store, handle).await?))
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
struct TimestampColumn {
column_id: u32,
name: String,
unit: TimestampUnit,
}
#[derive(Debug, Clone, Copy, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
enum TimestampUnit {
Second,
Millisecond,
Microsecond,
Nanosecond,
}
impl TimestampUnit {
fn arrow(self) -> TimeUnit {
match self {
Self::Second => TimeUnit::Second,
Self::Millisecond => TimeUnit::Millisecond,
Self::Microsecond => TimeUnit::Microsecond,
Self::Nanosecond => TimeUnit::Nanosecond,
}
}
fn from_arrow(unit: &TimeUnit) -> Self {
match unit {
TimeUnit::Second => Self::Second,
TimeUnit::Millisecond => Self::Millisecond,
TimeUnit::Microsecond => Self::Microsecond,
TimeUnit::Nanosecond => Self::Nanosecond,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
struct ValueColumn {
column_id: u32,
name: String,
#[serde(rename = "type")]
data_type: String,
nullable: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
struct Part {
part_id: u32,
file_size: u64,
num_rows: u64,
num_blocks: u64,
first: MetricSeriesId,
last: MetricSeriesId,
min_timestamp: i64,
max_timestamp: i64,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
struct Manifest {
format: ArtifactFormat,
version: u32,
region_id: u64,
source_file_id: FileId,
generation: FileId,
timestamp: TimestampColumn,
fields: Vec<ValueColumn>,
files: Vec<Part>,
}
impl Manifest {
fn prefix(&self) -> String {
format!(
"{}/samples/{}/{}/",
self.region_id, self.source_file_id, self.generation
)
}
fn path(&self) -> String {
format!("{}manifest.json", self.prefix())
}
fn part_path(&self, id: u32) -> String {
format!("{}part-{id}.samples", self.prefix())
}
fn validate_schema(&self) -> Result<()> {
require(
(1..=MAX_FIELDS).contains(&self.fields.len()),
"invalid samples field count",
)?;
let mut names = HashSet::from([TABLE_ID_COLUMN, TSID_COLUMN, SEQUENCE, OPERATION]);
require(
names.insert(&self.timestamp.name),
"duplicate samples timestamp name",
)?;
let mut ids = HashSet::from([self.timestamp.column_id]);
for field in &self.fields {
require(
field.data_type == "float64" && field.nullable,
"unsupported samples field type/nullability",
)?;
require(
names.insert(&field.name) && ids.insert(field.column_id),
"duplicate samples field name/id",
)?;
}
Ok(())
}
fn schema(&self) -> SchemaRef {
let mut fields = vec![
Field::new(TABLE_ID_COLUMN, DataType::UInt32, false),
Field::new(TSID_COLUMN, DataType::UInt64, false),
Field::new(
&self.timestamp.name,
DataType::Timestamp(self.timestamp.unit.arrow(), None),
false,
),
Field::new(SEQUENCE, DataType::UInt64, false),
Field::new(OPERATION, DataType::UInt8, false),
];
fields.extend(
self.fields
.iter()
.map(|f| Field::new(&f.name, DataType::Float64, true)),
);
Arc::new(Schema::new(fields))
}
}
fn valid_rows(rows: usize, fields: usize) -> bool {
(1..=MAX_ROWS).contains(&rows)
&& (1..=MAX_FIELDS).contains(&fields)
&& rows * (17 + 8 * fields) + fields * rows.div_ceil(8) <= BLOCK_BYTES
}
src/mito2/src/series_index/samples/codec.rs
+225
View File
@@ -0,0 +1,225 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Lossless, bounded sample column encodings. Nulls never become NaNs.
use crate::error::Result;
pub(super) use crate::series_index::columnar::codec::{
Cursor, checksum, decompress, put_u32, put_u64, put_varint,
};
use crate::series_index::samples::{MAX_ROWS, require, unsupported};
fn zig(x: i64) -> u64 {
((x as u64) << 1) ^ ((x >> 63) as u64)
}
fn unzig(x: u64) -> u64 {
(x >> 1) ^ (0u64.wrapping_sub(x & 1))
}
pub(super) fn encode_time(values: &[i64]) -> (u16, Vec<u8>) {
let mut out = values[0].to_le_bytes().to_vec();
let step = if values.len() == 1 {
Some(0)
} else {
values[1].checked_sub(values[0])
};
if let Some(step) = step.filter(|&step| {
step >= 0
&& values
.windows(2)
.all(|v| v[1].checked_sub(v[0]) == Some(step))
}) {
put_u64(&mut out, step as u64);
(1, out)
} else {
let mut prior = 0u64;
for (i, pair) in values.windows(2).enumerate() {
let delta = (pair[1] as u64).wrapping_sub(pair[0] as u64);
put_varint(
&mut out,
zig(if i == 0 {
delta
} else {
delta.wrapping_sub(prior)
} as i64),
);
prior = delta;
}
(2, out)
}
}
pub(super) fn encode_sequence(values: &[u64]) -> (u16, Vec<u8>) {
let mut out = values[0].to_le_bytes().to_vec();
if values.iter().all(|v| *v == values[0]) {
return (3, out);
}
for pair in values.windows(2) {
put_varint(&mut out, zig(pair[1].wrapping_sub(pair[0]) as i64));
}
(4, out)
}
pub(super) fn decode_numbers(bytes: &[u8], encoding: u16, rows: usize) -> Result<Vec<u64>> {
require(
(1..=MAX_ROWS).contains(&rows),
"invalid samples number count",
)?;
if !(1..=4).contains(&encoding) {
return Err(unsupported("samples number encoding"));
}
let mut c = Cursor::new(bytes);
let first = c.u64()?;
let mut values = Vec::with_capacity(rows);
values.push(first);
if encoding == 1 {
let step = c.u64()? as i64;
require(
step >= 0 && (rows != 1 || step == 0),
"invalid sample timestamp step",
)?;
let mut last = first as i64;
for _ in 1..rows {
last = last
.checked_add(step)
.ok_or_else(|| super::invalid("sample timestamp overflow"))?;
values.push(last as u64);
}
} else if encoding == 3 {
values.resize(rows, first);
} else {
require(
rows - 1 <= c.remaining(),
"sample number count exceeds bytes",
)?;
let mut previous = first;
let mut delta = 0u64;
for i in 1..rows {
let next = unzig(c.varint()?);
delta = if encoding == 4 || i == 1 {
next
} else {
delta.wrapping_add(next)
};
previous = previous.wrapping_add(delta);
values.push(previous);
}
}
c.end()?;
if encoding <= 2 {
require(
values.windows(2).all(|v| (v[0] as i64) <= v[1] as i64),
"unordered sample timestamps",
)?;
}
Ok(values)
}
pub(super) fn encode_float(values: &[Option<u64>]) -> Vec<u8> {
let nullable = values.iter().any(Option::is_none);
let mut out = vec![u8::from(nullable)];
if nullable {
out.resize(1 + values.len().div_ceil(8), 0);
for (i, value) in values.iter().enumerate() {
if value.is_some() {
out[1 + i / 8] |= 1 << (i % 8);
}
}
}
let mut previous = None;
for &value in values.iter().flatten() {
if let Some(last) = previous {
put_varint(&mut out, value ^ last);
} else {
put_u64(&mut out, value);
}
previous = Some(value);
}
out
}
pub(super) fn decode_float(bytes: &[u8], rows: usize, nulls: u32) -> Result<Vec<Option<u64>>> {
require(
(1..=MAX_ROWS).contains(&rows) && (nulls as usize) < rows,
"invalid samples float count",
)?;
let mut c = Cursor::new(bytes);
let bitmap = match c.u8()? {
0 => {
require(nulls == 0, "samples float null count mismatch")?;
None
}
1 => {
require(nulls > 0, "redundant samples validity bitmap")?;
let bitmap = c.take(rows.div_ceil(8))?;
if !rows.is_multiple_of(8) {
require(
bitmap[bitmap.len() - 1] >> (rows % 8) == 0,
"nonzero sample validity padding",
)?;
}
let present: u32 = bitmap.iter().map(|b| b.count_ones()).sum();
require(
present as usize + nulls as usize == rows,
"samples validity count mismatch",
)?;
Some(bitmap)
}
_ => return Err(unsupported("samples validity kind")),
};
let first = c.u64()?;
require(
rows - nulls as usize - 1 <= c.remaining(),
"sample float count exceeds bytes",
)?;
let mut previous = None;
let mut out = Vec::with_capacity(rows);
for i in 0..rows {
if bitmap.is_none_or(|b| b[i / 8] & (1 << (i % 8)) != 0) {
let value = match previous {
None => first,
Some(last) => last ^ c.varint()?,
};
out.push(Some(value));
previous = Some(value);
} else {
out.push(None);
}
}
c.end()?;
Ok(out)
}
pub(super) fn encode_operations(values: &[u8]) -> (u16, Vec<u8>) {
if values.iter().all(|v| *v == values[0]) {
(6, vec![values[0]])
} else {
(7, values.to_vec())
}
}
pub(super) fn decode_operations(bytes: &[u8], encoding: u16, rows: usize) -> Result<Vec<u8>> {
require(
(1..=MAX_ROWS).contains(&rows),
"invalid samples operation count",
)?;
let values = match encoding {
6 => {
require(bytes.len() == 1, "invalid constant sample operation")?;
vec![bytes[0]; rows]
}
7 => {
require(bytes.len() == rows, "invalid sample operation length")?;
bytes.to_vec()
}
_ => return Err(unsupported("samples operation encoding")),
};
require(values.iter().all(|v| *v <= 1), "unknown sample operation")?;
Ok(values)
}
src/mito2/src/series_index/samples/metadata.rs
+500
View File
@@ -0,0 +1,500 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Explicit sample framing and bounded two-level metadata pages.
use crate::error::Result;
use crate::series_index::samples::codec::{self, Cursor, put_u32, put_u64};
use crate::series_index::samples::{
Manifest, MetricSeriesId, VERSION, invalid, require, unsupported, valid_rows,
};
pub(super) const MAGIC: &[u8; 8] = b"GTSAMPLE";
pub(super) const HEADER_SIZE: u64 = 16;
pub(super) const TRAILER_SIZE: u64 = 112;
pub(super) const PAGE_LIMIT: usize = 65536;
pub(super) const ROOT_CAPACITY: usize = (PAGE_LIMIT - 12) / 72;
const CHUNK_LIMIT: usize = 1024 * 1024;
#[derive(Debug, Clone, Copy)]
pub(super) struct PayloadReference {
pub offset: u64,
pub stored: u32,
pub decoded: u32,
pub checksum: u64,
pub encoding: u16,
pub compression: u16,
}
impl PayloadReference {
pub fn end(&self) -> Result<u64> {
self.offset
.checked_add(u64::from(self.stored))
.ok_or_else(|| invalid("sample payload offset overflow"))
}
pub fn validate(&self, start: u64, end: u64, page: bool) -> Result<()> {
let limit = if page { PAGE_LIMIT } else { CHUNK_LIMIT };
require(
self.stored > 0
&& self.decoded > 0
&& self.decoded as usize <= limit
&& self.stored as usize <= limit + 65536,
"sample payload exceeds limits",
)?;
require(
self.offset >= start && self.end()? <= end,
"sample payload outside area",
)?;
if self.compression == 0 {
require(self.stored == self.decoded, "raw samples length mismatch")?;
}
Ok(())
}
fn encode(&self, out: &mut Vec<u8>) {
put_u64(out, self.offset);
put_u32(out, self.stored);
put_u32(out, self.decoded);
put_u64(out, self.checksum);
out.extend(self.encoding.to_le_bytes());
out.extend(self.compression.to_le_bytes());
put_u32(out, 0);
}
fn decode(c: &mut Cursor<'_>) -> Result<Self> {
let result = Self {
offset: c.u64()?,
stored: c.u32()?,
decoded: c.u32()?,
checksum: c.u64()?,
encoding: c.u16()?,
compression: c.u16()?,
};
require(c.u32()? == 0, "nonzero sample payload reserved field")?;
Ok(result)
}
pub fn unpack(&self, bytes: &[u8]) -> Result<Vec<u8>> {
require(
bytes.len() == self.stored as usize && codec::checksum(bytes) == self.checksum,
"sample checksum mismatch",
)?;
codec::decompress(bytes, self.compression, self.decoded as usize)
}
}
pub(super) fn pack(
bytes: &[u8],
offset: u64,
encoding: u16,
page: bool,
) -> Result<(PayloadReference, Vec<u8>)> {
require(
!bytes.is_empty() && bytes.len() <= if page { PAGE_LIMIT } else { CHUNK_LIMIT },
"sample encoded payload exceeds limit",
)?;
let compressed =
zstd::bulk::compress(bytes, 3).map_err(|e| invalid(format!("sample compression: {e}")))?;
let (compression, stored) = if compressed.len() < bytes.len() {
(1, compressed)
} else {
(0, bytes.to_vec())
};
let reference = PayloadReference {
offset,
stored: stored.len() as u32,
decoded: bytes.len() as u32,
checksum: codec::checksum(&stored),
encoding,
compression,
};
reference.validate(offset, reference.end()?, page)?;
Ok((reference, stored))
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub(super) struct Bounds {
pub first: MetricSeriesId,
pub last: MetricSeriesId,
pub min: i64,
pub max: i64,
}
impl Bounds {
pub fn validate(&self) -> Result<()> {
require(
self.first <= self.last && self.min <= self.max,
"reversed sample bounds",
)
}
pub fn extend(&mut self, next: Self) {
self.last = next.last;
self.min = self.min.min(next.min);
self.max = self.max.max(next.max);
}
fn encode(&self, out: &mut Vec<u8>) {
for id in [self.first, self.last] {
put_u32(out, id.table_id);
put_u64(out, id.tsid);
}
put_u64(out, self.min as u64);
put_u64(out, self.max as u64);
}
fn decode(c: &mut Cursor<'_>) -> Result<Self> {
let result = Self {
first: MetricSeriesId {
table_id: c.u32()?,
tsid: c.u64()?,
},
last: MetricSeriesId {
table_id: c.u32()?,
tsid: c.u64()?,
},
min: c.u64()? as i64,
max: c.u64()? as i64,
};
result.validate()?;
Ok(result)
}
}
#[derive(Debug, Clone)]
pub(super) struct PageReference {
pub bounds: Bounds,
pub payload: PayloadReference,
}
impl PageReference {
fn encode(&self, out: &mut Vec<u8>) {
self.bounds.encode(out);
self.payload.encode(out);
}
fn decode(c: &mut Cursor<'_>) -> Result<Self> {
let reference = Self {
bounds: Bounds::decode(c)?,
payload: PayloadReference::decode(c)?,
};
if reference.payload.encoding != 1 || reference.payload.compression > 1 {
return Err(unsupported("samples metadata codec"));
}
Ok(reference)
}
}
#[derive(Debug, Clone)]
pub(super) struct Column {
pub role: u32,
pub id: u32,
pub payload: PayloadReference,
pub nulls: u32,
}
#[derive(Debug, Clone)]
pub(super) struct Block {
pub id: MetricSeriesId,
pub min: i64,
pub max: i64,
pub ordinal: u64,
pub rows: u32,
pub columns: Vec<Column>,
}
impl Block {
pub fn bounds(&self) -> Bounds {
Bounds {
first: self.id,
last: self.id,
min: self.min,
max: self.max,
}
}
pub fn size(&self) -> usize {
44 + 44 * self.columns.len()
}
pub fn encode(&self, out: &mut Vec<u8>) {
put_u32(out, self.id.table_id);
put_u64(out, self.id.tsid);
put_u64(out, self.min as u64);
put_u64(out, self.max as u64);
put_u64(out, self.ordinal);
put_u32(out, self.rows);
put_u32(out, self.columns.len() as u32);
for column in &self.columns {
put_u32(out, column.role);
put_u32(out, column.id);
column.payload.encode(out);
put_u32(out, column.nulls);
}
}
fn decode(
c: &mut Cursor<'_>,
manifest: &Manifest,
payload_end: u64,
previous_end: &mut u64,
) -> Result<Self> {
let id = MetricSeriesId {
table_id: c.u32()?,
tsid: c.u64()?,
};
let min = c.u64()? as i64;
let max = c.u64()? as i64;
let ordinal = c.u64()?;
let rows = c.u32()?;
let count = c.u32()? as usize;
require(
min <= max && valid_rows(rows as usize, manifest.fields.len()),
"invalid sample block bounds/count",
)?;
require(
count >= 3 && count <= manifest.fields.len() + 3 && count <= c.remaining() / 44,
"invalid sample column count",
)?;
let mut columns = Vec::with_capacity(count);
let mut next_field = 0;
for index in 0..count {
let role = c.u32()?;
let column_id = c.u32()?;
let payload = PayloadReference::decode(c)?;
let nulls = c.u32()?;
if index < 3 {
require(
role == index as u32 + 1
&& column_id
== if index == 0 {
manifest.timestamp.column_id
} else {
0
}
&& nulls == 0,
"invalid mandatory sample column",
)?;
} else {
require(
role == 4 && nulls < rows,
"invalid sample field role/null count",
)?;
let position = manifest.fields[next_field..]
.iter()
.position(|f| f.column_id == column_id)
.ok_or_else(|| invalid("unknown or unordered sample field"))?
+ next_field;
next_field = position + 1;
}
if payload.encoding <= 7 {
let valid = match role {
1 => matches!(payload.encoding, 1 | 2),
2 => matches!(payload.encoding, 3 | 4),
3 => matches!(payload.encoding, 6 | 7),
4 => payload.encoding == 5,
_ => false,
};
require(valid, "sample codec role mismatch")?;
}
payload.validate(*previous_end, payload_end, false)?;
*previous_end = payload.end()?;
columns.push(Column {
role,
id: column_id,
payload,
nulls,
});
}
Ok(Self {
id,
min,
max,
ordinal,
rows,
columns,
})
}
pub fn follows(&self, previous: &Self, adjacent: bool) -> Result<()> {
require(previous.id <= self.id, "unordered sample identifiers")?;
if self.id == previous.id {
require(
previous.max <= self.min && previous.ordinal < self.ordinal,
"unordered sample blocks",
)?;
if adjacent {
require(
previous.ordinal.checked_add(1) == Some(self.ordinal),
"nonconsecutive sample ordinal",
)?;
}
} else if adjacent {
require(self.ordinal == 0, "first sample block has nonzero ordinal")?;
}
Ok(())
}
}
#[derive(Debug)]
pub(super) enum Page {
Leaf(Vec<Block>),
Internal(Vec<PageReference>),
}
impl Page {
pub fn bounds(&self) -> Result<Bounds> {
let bounds: Vec<_> = match self {
Self::Leaf(v) => v.iter().map(Block::bounds).collect(),
Self::Internal(v) => v.iter().map(|c| c.bounds).collect(),
};
let mut result = *bounds.first().ok_or_else(|| invalid("empty sample page"))?;
for bound in &bounds[1..] {
result.extend(*bound);
}
Ok(result)
}
pub fn encode(&self) -> Vec<u8> {
let mut out = Vec::new();
match self {
Self::Leaf(v) => {
put_u32(&mut out, 0);
put_u64(&mut out, v.len() as u64);
for block in v {
block.encode(&mut out);
}
}
Self::Internal(v) => {
put_u32(&mut out, 1);
put_u64(&mut out, v.len() as u64);
for child in v {
child.encode(&mut out);
}
}
}
out
}
pub fn decode(
bytes: &[u8],
reference: &PageReference,
depth: u32,
payload_end: u64,
manifest: &Manifest,
) -> Result<Self> {
require(bytes.len() <= PAGE_LIMIT, "sample page exceeds limit")?;
let mut c = Cursor::new(bytes);
let kind = c.u32()?;
let count = c.length()?;
require(count > 0, "empty sample page")?;
let page = match (kind, depth) {
(0, 1) => {
require(
count <= c.remaining() / 176,
"sample block count exceeds page",
)?;
let mut blocks: Vec<Block> = Vec::with_capacity(count);
let mut end = HEADER_SIZE;
for _ in 0..count {
let block = Block::decode(&mut c, manifest, payload_end, &mut end)?;
if let Some(previous) = blocks.last() {
block.follows(previous, true)?;
}
blocks.push(block);
}
Self::Leaf(blocks)
}
(1, 2) => {
require(
count <= ROOT_CAPACITY && count <= c.remaining() / 72,
"sample root exceeds capacity",
)?;
let mut children: Vec<PageReference> = Vec::with_capacity(count);
let mut end = payload_end;
for _ in 0..count {
let child = PageReference::decode(&mut c)?;
child
.payload
.validate(end, reference.payload.offset, true)?;
end = child.payload.end()?;
if let Some(previous) = children.last() {
require(
previous.bounds.last <= child.bounds.first,
"unordered sample child ranges",
)?;
}
children.push(child);
}
Self::Internal(children)
}
_ => return Err(unsupported("samples metadata kind/depth")),
};
c.end()?;
require(
page.bounds()? == reference.bounds,
"samples page bounds disagree with parent",
)?;
Ok(page)
}
}
pub(super) fn header() -> Vec<u8> {
let mut out = MAGIC.to_vec();
put_u32(&mut out, VERSION);
put_u32(&mut out, 0);
out
}
pub(super) fn validate_header(bytes: &[u8]) -> Result<()> {
let mut c = Cursor::new(bytes);
require(c.take(8)? == MAGIC, "sample header magic")?;
if c.u32()? != VERSION {
return Err(unsupported("samples file version"));
}
require(c.u32()? == 0, "nonzero sample header reserved field")?;
c.end()
}
pub(super) struct Trailer {
pub root: PageReference,
pub depth: u32,
pub file_length: u64,
pub payload_end: u64,
}
impl Trailer {
pub fn encode(&self) -> Vec<u8> {
let mut out = Vec::new();
self.root.encode(&mut out);
put_u32(&mut out, self.depth);
put_u32(&mut out, VERSION);
put_u64(&mut out, self.file_length);
put_u64(&mut out, self.payload_end);
out.extend(MAGIC);
let sum = codec::checksum(&out);
put_u64(&mut out, sum);
out
}
pub fn decode(bytes: &[u8], file_length: u64) -> Result<Self> {
require(
bytes.len() == TRAILER_SIZE as usize,
"truncated sample trailer",
)?;
let mut c = Cursor::new(bytes);
let root = PageReference::decode(&mut c)?;
let depth = c.u32()?;
let version = c.u32()?;
if !(1..=2).contains(&depth) || version != VERSION {
return Err(unsupported("sample trailer version/depth"));
}
let length = c.u64()?;
let payload_end = c.u64()?;
require(
c.take(8)? == MAGIC && c.u64()? == codec::checksum(&bytes[..104]),
"sample trailer checksum/magic",
)?;
require(
length == file_length
&& payload_end >= HEADER_SIZE
&& file_length >= HEADER_SIZE + TRAILER_SIZE,
"invalid sample framing",
)?;
root.payload
.validate(payload_end, file_length - TRAILER_SIZE, true)?;
require(
root.payload.end()? == file_length - TRAILER_SIZE,
"sample root must end at trailer",
)?;
Ok(Self {
root,
depth,
file_length,
payload_end,
})
}
}
src/mito2/src/series_index/samples/reader.rs
+418
View File
@@ -0,0 +1,418 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Lazy, selective sample reads with independent traversal and pinned lifetimes.
use std::collections::{BTreeMap, HashMap, HashSet};
use std::sync::Arc;
use datatypes::arrow::array::{
ArrayRef, Float64Array, TimestampMicrosecondArray, TimestampMillisecondArray,
TimestampNanosecondArray, TimestampSecondArray, UInt8Array, UInt32Array, UInt64Array,
};
use datatypes::arrow::datatypes::SchemaRef;
use datatypes::arrow::record_batch::{RecordBatch, RecordBatchOptions};
use futures::stream::BoxStream;
use object_store::ObjectStore;
use snafu::ResultExt;
use crate::error::{NewRecordBatchSnafu, OpenDalSnafu, Result, SerdeJsonSnafu};
use crate::series_index::artifact::{ArtifactFormat, SamplesFileHandle};
use crate::series_index::format::{SamplesFormatReader, SamplesReadRequest};
use crate::series_index::samples::codec;
use crate::series_index::samples::metadata::{
self, Block, Bounds, Column, Page, PageReference, Trailer,
};
use crate::series_index::samples::{
MANIFEST_LIMIT, Manifest, MetricSeriesId, TimestampUnit, VERSION, invalid, require, unsupported,
};
use crate::sst::parquet::helper::fetch_byte_ranges;
pub(super) struct SamplesReader {
store: ObjectStore,
manifest: Arc<Manifest>,
handle: SamplesFileHandle,
}
impl SamplesReader {
pub async fn open(store: ObjectStore, handle: SamplesFileHandle) -> Result<Self> {
let entry = handle.entry();
let artifact = &entry.artifact;
require(
artifact.format == ArtifactFormat::ColumnarSamples,
"samples artifact format mismatch",
)?;
let region = handle.file_id().region_id().as_u64();
require(
handle.file_id().file_id() == entry.file_id,
"samples source handle mismatch",
)?;
let path = format!(
"{region}/samples/{}/{}/manifest.json",
entry.file_id, artifact.generation
);
let mut objects = HashMap::new();
for object in &artifact.objects {
require(
objects
.insert(object.path.as_str(), object.file_size)
.is_none(),
"duplicate sample artifact object",
)?;
}
let length = *objects
.get(path.as_str())
.ok_or_else(|| invalid("missing samples manifest"))?;
require(
length > 0 && length <= MANIFEST_LIMIT as u64,
"samples manifest exceeds limit",
)?;
let bytes = read_range(&store, &path, 0, length).await?;
let manifest: Manifest = serde_json::from_slice(&bytes).context(SerdeJsonSnafu)?;
if manifest.version != VERSION {
return Err(unsupported("samples manifest version"));
}
require(
manifest.format == artifact.format
&& manifest.region_id == region
&& manifest.source_file_id == entry.file_id
&& manifest.generation == artifact.generation,
"samples manifest identity mismatch",
)?;
manifest.validate_schema()?;
let mut paths = HashSet::from([path]);
let mut total = length;
let mut previous = None;
for (index, part) in manifest.files.iter().enumerate() {
require(
index as u64 == u64::from(part.part_id)
&& part.num_rows > 0
&& part.num_blocks > 0
&& part.num_blocks <= part.num_rows
&& part.first <= part.last
&& part.min_timestamp <= part.max_timestamp
&& part.file_size >= metadata::HEADER_SIZE + metadata::TRAILER_SIZE,
"invalid sample part descriptor",
)?;
if let Some(last) = previous {
require(last <= part.first, "unordered sample parts")?;
}
previous = Some(part.last);
let part_path = manifest.part_path(part.part_id);
require(
objects.get(part_path.as_str()) == Some(&part.file_size),
"sample part size mismatch",
)?;
paths.insert(part_path);
total = total
.checked_add(part.file_size)
.ok_or_else(|| invalid("sample artifact size overflow"))?;
}
require(
total == artifact.file_size
&& paths.len() == artifact.objects.len()
&& artifact.objects.iter().all(|o| paths.contains(&o.path)),
"sample artifact objects/size mismatch",
)?;
Ok(Self {
store,
manifest: Arc::new(manifest),
handle,
})
}
}
impl SamplesFormatReader for SamplesReader {
fn schema(&self) -> SchemaRef {
self.manifest.schema()
}
fn read(&self, request: SamplesReadRequest) -> Result<BoxStream<'static, Result<RecordBatch>>> {
let schema = self.schema();
let mut projection = Vec::new();
let mut seen = HashSet::new();
for name in &request.projection {
require(seen.insert(name), "duplicate sample projection")?;
projection.push(
schema
.index_of(name)
.map_err(|_| invalid(format!("unknown sample projection {name}")))?,
);
}
require(
request
.start
.zip(request.end)
.is_none_or(|(start, end)| start <= end),
"reversed sample time range",
)?;
let empty = request.ids.as_ref().is_some_and(|ids| ids.is_empty())
|| request.start.zip(request.end).is_some_and(|(a, b)| a == b);
let complete = request.ids.is_none()
&& request.table_id.is_none()
&& request.start.is_none()
&& request.end.is_none();
let projected = Arc::new(
schema
.project(&projection)
.map_err(|e| invalid(format!("sample projection: {e}")))?,
);
let manifest = self.manifest.clone();
let store = self.store.clone();
let handle = self.handle.clone();
Ok(Box::pin(async_stream::try_stream! {
let _handle = handle;
if empty { return; }
let mut previous: Option<Block> = None;
let mut adjacent = true;
for part in &manifest.files {
let bounds = Bounds { first: part.first, last: part.last, min: part.min_timestamp, max: part.max_timestamp };
if !matches(bounds, &request) { adjacent = false; continue; }
let path = manifest.part_path(part.part_id);
let header = read_range(&store, &path, 0, metadata::HEADER_SIZE).await?;
metadata::validate_header(&header)?;
let bytes = read_range(&store, &path, part.file_size - metadata::TRAILER_SIZE, part.file_size).await?;
let trailer = Trailer::decode(&bytes, part.file_size)?;
require(trailer.root.bounds == bounds, "sample root/part bounds mismatch")?;
let root = load_metadata_page(&store, &path, &trailer.root, trailer.depth, trailer.payload_end, &manifest).await?;
let (mut active, children) = match root { Page::Leaf(blocks) => (Some(blocks), Vec::new()), Page::Internal(children) => (None, children) };
let mut children = children.into_iter(); let mut row_count = 0u64; let mut block_count = 0u64;
let mut payload_position = metadata::HEADER_SIZE;
loop {
let blocks = if let Some(blocks) = active.take() { blocks } else {
let Some(child) = children.next() else { break; };
if !matches(child.bounds, &request) { adjacent = false; continue; }
match load_metadata_page(&store, &path, &child, 1, trailer.payload_end, &manifest).await? {
Page::Leaf(blocks) => blocks,
Page::Internal(_) => Err(unsupported("internal child in sample depth-two tree"))?,
}
};
for block in blocks {
row_count = row_count.checked_add(u64::from(block.rows)).ok_or_else(|| invalid("samples row count overflow"))?;
block_count += 1;
if let Some(prior) = &previous { block.follows(prior, adjacent)?; }
else if complete { require(block.ordinal == 0, "initial sample ordinal is not zero")?; }
for column in &block.columns {
column.payload.validate(payload_position, trailer.payload_end, false)?;
payload_position = column.payload.end()?;
}
let batch = if matches(block.bounds(), &request) {
read_block(&store, &path, &block, &manifest, &request, &projection, projected.clone()).await?
} else { None };
if let Some(batch) = batch { yield batch; }
previous = Some(block); adjacent = true;
}
}
if complete { require(row_count == part.num_rows && block_count == part.num_blocks, "sample part totals mismatch")?; }
}
}))
}
}
fn matches(bounds: Bounds, request: &SamplesReadRequest) -> bool {
if request.start.is_some_and(|start| bounds.max < start)
|| request.end.is_some_and(|end| bounds.min >= end)
{
return false;
}
let mut first = bounds.first;
let mut last = bounds.last;
if let Some(table_id) = request.table_id {
first = first.max(MetricSeriesId { table_id, tsid: 0 });
last = last.min(MetricSeriesId {
table_id,
tsid: u64::MAX,
});
}
first <= last
&& request
.ids
.as_ref()
.is_none_or(|ids| ids.range(first..=last).next().is_some())
}
async fn read_range(store: &ObjectStore, path: &str, start: u64, end: u64) -> Result<bytes::Bytes> {
require(start <= end, "reversed sample byte range")?;
let mut buffers = fetch_byte_ranges(path, store.clone(), std::slice::from_ref(&(start..end)))
.await
.context(OpenDalSnafu)?;
let bytes = buffers
.pop()
.ok_or_else(|| invalid("missing sample range response"))?;
require(
buffers.is_empty() && bytes.len() as u64 == end - start,
"sample range length mismatch",
)?;
Ok(bytes)
}
/// Cache integration can key this entrypoint by immutable generation, range and codecs.
async fn load_metadata_page(
store: &ObjectStore,
path: &str,
reference: &PageReference,
depth: u32,
payload_end: u64,
manifest: &Manifest,
) -> Result<Page> {
if reference.payload.encoding != 1 {
return Err(unsupported("sample metadata codec"));
}
let bytes = read_range(
store,
path,
reference.payload.offset,
reference.payload.end()?,
)
.await?;
Page::decode(
&reference.payload.unpack(&bytes)?,
reference,
depth,
payload_end,
manifest,
)
}
/// Keep column loads separate from metadata for future Mito cache integration.
async fn load_column_chunk(store: &ObjectStore, path: &str, column: &Column) -> Result<Vec<u8>> {
let allowed = match column.role {
1 => matches!(column.payload.encoding, 1 | 2),
2 => matches!(column.payload.encoding, 3 | 4),
3 => matches!(column.payload.encoding, 6 | 7),
4 => column.payload.encoding == 5,
_ => false,
};
if !allowed {
return Err(unsupported("samples column encoding"));
}
let bytes = read_range(store, path, column.payload.offset, column.payload.end()?).await?;
column.payload.unpack(&bytes)
}
fn timestamp_array(values: impl Iterator<Item = i64>, unit: TimestampUnit) -> ArrayRef {
match unit {
TimestampUnit::Second => Arc::new(TimestampSecondArray::from_iter_values(values)),
TimestampUnit::Millisecond => Arc::new(TimestampMillisecondArray::from_iter_values(values)),
TimestampUnit::Microsecond => Arc::new(TimestampMicrosecondArray::from_iter_values(values)),
TimestampUnit::Nanosecond => Arc::new(TimestampNanosecondArray::from_iter_values(values)),
}
}
async fn read_block(
store: &ObjectStore,
path: &str,
block: &Block,
manifest: &Manifest,
request: &SamplesReadRequest,
projection: &[usize],
schema: SchemaRef,
) -> Result<Option<RecordBatch>> {
let mut selected: Vec<usize> = (0..block.rows as usize).collect();
let mut times = None;
if request.start.is_some() || request.end.is_some() || projection.contains(&2) {
let column = &block.columns[0];
let values = codec::decode_numbers(
&load_column_chunk(store, path, column).await?,
column.payload.encoding,
block.rows as usize,
)?;
require(
values.first().map(|v| *v as i64) == Some(block.min)
&& values.last().map(|v| *v as i64) == Some(block.max),
"sample timestamp bounds mismatch",
)?;
selected.retain(|&i| {
request
.start
.is_none_or(|start| (values[i] as i64) >= start)
&& request.end.is_none_or(|end| (values[i] as i64) < end)
});
if selected.is_empty() {
return Ok(None);
}
times = Some(values);
}
let mut arrays: BTreeMap<usize, ArrayRef> = BTreeMap::new();
// Visit columns in physical order, regardless of requested output order.
let mut sorted = projection.to_vec();
sorted.sort_unstable();
for index in sorted {
let array: ArrayRef = match index {
0 => Arc::new(UInt32Array::from(vec![block.id.table_id; selected.len()])),
1 => Arc::new(UInt64Array::from(vec![block.id.tsid; selected.len()])),
2 => {
let values = times
.as_ref()
.ok_or_else(|| invalid("missing decoded sample time"))?;
timestamp_array(
selected.iter().map(|&i| values[i] as i64),
manifest.timestamp.unit,
)
}
3 => {
let column = &block.columns[1];
let values = codec::decode_numbers(
&load_column_chunk(store, path, column).await?,
column.payload.encoding,
block.rows as usize,
)?;
if let Some(times) = &times {
require(
(1..values.len())
.all(|i| times[i - 1] != times[i] || values[i - 1] >= values[i]),
"unordered sample sequences",
)?;
}
Arc::new(UInt64Array::from_iter_values(
selected.iter().map(|&i| values[i]),
))
}
4 => {
let column = &block.columns[2];
let values = codec::decode_operations(
&load_column_chunk(store, path, column).await?,
column.payload.encoding,
block.rows as usize,
)?;
Arc::new(UInt8Array::from_iter_values(
selected.iter().map(|&i| values[i]),
))
}
field => {
let id = manifest.fields[field - 5].column_id;
if let Some(column) = block.columns.iter().find(|c| c.role == 4 && c.id == id) {
let values = codec::decode_float(
&load_column_chunk(store, path, column).await?,
block.rows as usize,
column.nulls,
)?;
Arc::new(Float64Array::from_iter(
selected.iter().map(|&i| values[i].map(f64::from_bits)),
))
} else {
Arc::new(Float64Array::from(vec![None::<f64>; selected.len()]))
}
}
};
arrays.insert(index, array);
}
let output = projection
.iter()
.map(|i| {
arrays
.remove(i)
.ok_or_else(|| invalid("missing sample projected column"))
})
.collect::<Result<Vec<_>>>()?;
Ok(Some(
RecordBatch::try_new_with_options(
schema,
output,
&RecordBatchOptions::new().with_row_count(Some(selected.len())),
)
.context(NewRecordBatchSnafu)?,
))
}
src/mito2/src/series_index/samples/tests.rs
+1065
View File
File diff suppressed because it is too large Load Diff
src/mito2/src/series_index/samples/writer.rs
+485
View File
@@ -0,0 +1,485 @@
// Copyright 2023 Greptime Team
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//! Streaming per-SST sample writing with bounded blocks and owned outputs.
use async_trait::async_trait;
use datatypes::arrow::array::{
Array, BinaryArray, DictionaryArray, Float64Array, UInt8Array, UInt64Array,
};
use datatypes::arrow::datatypes::{DataType, UInt32Type};
use datatypes::arrow::record_batch::RecordBatch;
use datatypes::prelude::DataType as _;
use datatypes::timestamp::timestamp_array_to_primitive;
use mito_codec::row_converter::SparseOffsetsCache;
use object_store::ObjectStore;
use snafu::ResultExt;
use store_api::metadata::RegionMetadataRef;
use store_api::storage::{FileId, consts};
use crate::error::{Result, SerdeJsonSnafu};
use crate::series_index::artifact::{ArtifactFormat, ArtifactMetadata, ArtifactObject};
use crate::series_index::artifact_writer::Outputs;
use crate::series_index::format::FormatWriter;
use crate::series_index::samples::codec;
use crate::series_index::samples::metadata::{
self, Block, Column, Page, PageReference, Trailer, pack,
};
use crate::series_index::samples::{
ColumnarSamplesFormat, MANIFEST_LIMIT, Manifest, MetricSeriesId, Part, TimestampColumn,
TimestampUnit, VERSION, ValueColumn, invalid, require, valid_rows,
};
use crate::series_index::writer::{decode_series_identity, validate_metadata};
struct Row {
id: MetricSeriesId,
time: i64,
sequence: u64,
operation: u8,
fields: Vec<Option<u64>>,
}
pub(super) struct SamplesWriter {
options: ColumnarSamplesFormat,
manifest: Manifest,
outputs: Outputs,
offsets: SparseOffsetsCache,
key_buffer: Vec<u8>,
rows: Vec<Row>,
last: Option<(MetricSeriesId, i64, u64)>,
ordinal: u64,
leaves: Vec<Vec<Block>>,
leaf_bytes: usize,
descriptor_bytes: usize,
position: u64,
part_rows: u64,
part_blocks: u64,
manifest_bytes: usize,
failed: bool,
}
impl SamplesWriter {
pub fn new(
options: ColumnarSamplesFormat,
metadata: RegionMetadataRef,
store: ObjectStore,
prefix: &str,
source_file_id: FileId,
generation: FileId,
) -> Result<Self> {
validate_metadata(&metadata)?;
require(
(1..=super::MAX_ROWS).contains(&options.block_rows)
&& (1..=metadata::ROOT_CAPACITY).contains(&options.root_capacity)
&& (188..=metadata::PAGE_LIMIT).contains(&options.page_bytes)
&& options.part_bytes > 0
&& options.metadata_bytes > 0,
"invalid samples packing options",
)?;
let timestamp = metadata.time_index_column();
let DataType::Timestamp(unit, None) = timestamp.column_schema.data_type.as_arrow_type()
else {
return Err(invalid("samples require a timestamp time index"));
};
let fields = metadata
.field_columns()
.map(|column| {
require(
column.column_schema.data_type.as_arrow_type() == DataType::Float64,
"samples support only Float64 fields",
)?;
Ok(ValueColumn {
column_id: column.column_id,
name: column.column_schema.name.clone(),
data_type: "float64".into(),
nullable: true,
})
})
.collect::<Result<Vec<_>>>()?;
let manifest = Manifest {
format: ArtifactFormat::ColumnarSamples,
version: VERSION,
region_id: metadata.region_id.as_u64(),
source_file_id,
generation,
timestamp: TimestampColumn {
column_id: timestamp.column_id,
name: timestamp.column_schema.name.clone(),
unit: TimestampUnit::from_arrow(&unit),
},
fields,
files: Vec::new(),
};
manifest.validate_schema()?;
require(
prefix == manifest.prefix(),
"samples output prefix mismatch",
)?;
let manifest_bytes = serde_json::to_vec(&manifest).context(SerdeJsonSnafu)?.len();
require(
manifest_bytes <= MANIFEST_LIMIT,
"samples schema exceeds manifest limit",
)?;
Ok(Self {
options,
manifest,
outputs: Outputs {
store,
active: None,
paths: Vec::new(),
committed: false,
},
offsets: SparseOffsetsCache::new(),
key_buffer: Vec::new(),
rows: Vec::new(),
last: None,
ordinal: 0,
leaves: Vec::new(),
leaf_bytes: 12,
descriptor_bytes: 0,
position: 0,
part_rows: 0,
part_blocks: 0,
manifest_bytes,
failed: false,
})
}
async fn write_batch(&mut self, batch: &RecordBatch) -> Result<()> {
require(batch.num_columns() >= 4, "samples require flat batches")?;
let column = |name: &str| {
batch
.column_by_name(name)
.ok_or_else(|| invalid(format!("missing sample input column {name}")))
};
let time = column(&self.manifest.timestamp.name)?;
require(
time.data_type() == &DataType::Timestamp(self.manifest.timestamp.unit.arrow(), None)
&& time.null_count() == 0,
"samples timestamp type/null mismatch",
)?;
let (times, _) = timestamp_array_to_primitive(time)
.ok_or_else(|| invalid("invalid sample timestamp array"))?;
let sequence = column(consts::SEQUENCE_COLUMN_NAME)?
.as_any()
.downcast_ref::<UInt64Array>()
.ok_or_else(|| invalid("invalid sample sequence type"))?;
let operations = column(consts::OP_TYPE_COLUMN_NAME)?
.as_any()
.downcast_ref::<UInt8Array>()
.ok_or_else(|| invalid("invalid sample operation type"))?;
require(
sequence.null_count() == 0 && operations.null_count() == 0,
"null sample sequence/operation",
)?;
let fields = self
.manifest
.fields
.iter()
.map(|field| {
column(&field.name)?
.as_any()
.downcast_ref::<Float64Array>()
.ok_or_else(|| invalid("invalid sample field array"))
})
.collect::<Result<Vec<_>>>()?;
let keys = column(consts::PRIMARY_KEY_COLUMN_NAME)?;
require(keys.null_count() == 0, "null sample primary key")?;
let binary = keys.as_any().downcast_ref::<BinaryArray>();
let dictionary = keys.as_any().downcast_ref::<DictionaryArray<UInt32Type>>();
let values = dictionary.and_then(|d| d.values().as_any().downcast_ref::<BinaryArray>());
require(
binary.is_some() || values.is_some(),
"samples require binary or UInt32 dictionary primary keys",
)?;
for i in 0..batch.num_rows() {
let key = if let Some(binary) = binary {
binary.value(i)
} else {
let dictionary = dictionary.ok_or_else(|| invalid("missing sample dictionary"))?;
let values = values.ok_or_else(|| invalid("invalid sample dictionary values"))?;
let index = dictionary.keys().value(i) as usize;
require(
index < values.len() && !values.is_null(index),
"invalid sample dictionary key",
)?;
values.value(index)
};
// TODO: coordinate exact keys and future job/instance identity components
// with the series format. This port assumes (table_id, tsid) is unique.
let (id, _) =
decode_series_identity(key, &[], &mut self.offsets, &mut self.key_buffer)?;
let time = times.value(i);
let seq = sequence.value(i);
let op = operations.value(i);
require(op <= 1, "unknown sample operation")?;
if let Some((last_id, last_time, last_seq)) = self.last {
require(
id >= last_id
&& (id != last_id
|| time > last_time
|| (time == last_time && seq <= last_seq)),
"sample input is not sorted",
)?;
if id != last_id {
self.flush_block().await?;
self.ordinal = 0;
}
}
if self.rows.len() == self.options.block_rows
|| !valid_rows(self.rows.len() + 1, fields.len())
{
self.flush_block().await?;
}
self.rows.push(Row {
id,
time,
sequence: seq,
operation: op,
fields: fields
.iter()
.map(|field| {
if field.is_null(i) {
None
} else {
Some(field.value(i).to_bits())
}
})
.collect(),
});
self.last = Some((id, time, seq));
}
Ok(())
}
async fn flush_block(&mut self) -> Result<()> {
if self.rows.is_empty() {
return Ok(());
}
let rows = std::mem::take(&mut self.rows);
let (mut block, chunks) = encode_block(&rows, &self.manifest, self.ordinal)?;
self.ordinal = self
.ordinal
.checked_add(1)
.ok_or_else(|| invalid("sample ordinal overflow"))?;
let size = block.size();
require(
size + 12 <= self.options.page_bytes,
"sample descriptor exceeds leaf limit",
)?;
if !self.leaves.is_empty()
&& self.leaf_bytes + size > self.options.page_bytes
&& self.leaves.len() == self.options.root_capacity
{
self.finish_part().await?;
}
if self.outputs.active.is_none() {
let id = u32::try_from(self.manifest.files.len())
.map_err(|_| invalid("too many sample parts"))?;
self.outputs.open(self.manifest.part_path(id)).await?;
self.outputs.write(metadata::header()).await?;
self.position = metadata::HEADER_SIZE;
}
for (column, chunk) in block.columns.iter_mut().zip(chunks) {
column.payload.offset = self.position;
self.position = column.payload.end()?;
self.outputs.write(chunk).await?;
}
if self.leaves.is_empty() || self.leaf_bytes + size > self.options.page_bytes {
self.leaves.push(Vec::new());
self.leaf_bytes = 12;
}
self.leaf_bytes += size;
self.descriptor_bytes += size;
self.part_rows = self
.part_rows
.checked_add(u64::from(block.rows))
.ok_or_else(|| invalid("sample row count overflow"))?;
self.part_blocks += 1;
self.leaves
.last_mut()
.ok_or_else(|| invalid("missing sample leaf"))?
.push(block);
if self.position >= self.options.part_bytes
|| self.descriptor_bytes >= self.options.metadata_bytes
{
self.finish_part().await?;
}
Ok(())
}
async fn write_page(&mut self, page: Page) -> Result<PageReference> {
let bounds = page.bounds()?;
let (payload, bytes) = pack(&page.encode(), self.position, 1, true)?;
self.position = payload.end()?;
self.outputs.write(bytes).await?;
Ok(PageReference { bounds, payload })
}
async fn finish_part(&mut self) -> Result<()> {
if self.leaves.is_empty() {
return Ok(());
}
let payload_end = self.position;
let mut children = Vec::new();
for leaf in std::mem::take(&mut self.leaves) {
children.push(self.write_page(Page::Leaf(leaf)).await?);
}
let depth = if children.len() == 1 { 1 } else { 2 };
let root = if depth == 1 {
children.remove(0)
} else {
self.write_page(Page::Internal(children)).await?
};
let bounds = root.bounds;
let file_length = self
.position
.checked_add(metadata::TRAILER_SIZE)
.ok_or_else(|| invalid("sample file length overflow"))?;
self.outputs
.write(
Trailer {
root,
depth,
file_length,
payload_end,
}
.encode(),
)
.await?;
self.outputs.close().await?;
let part = Part {
part_id: u32::try_from(self.manifest.files.len())
.map_err(|_| invalid("too many sample parts"))?,
file_size: file_length,
num_rows: self.part_rows,
num_blocks: self.part_blocks,
first: bounds.first,
last: bounds.last,
min_timestamp: bounds.min,
max_timestamp: bounds.max,
};
self.manifest_bytes += serde_json::to_vec(&part).context(SerdeJsonSnafu)?.len()
+ usize::from(!self.manifest.files.is_empty());
require(
self.manifest_bytes <= MANIFEST_LIMIT,
"samples manifest exceeds size limit",
)?;
self.manifest.files.push(part);
self.position = 0;
self.part_rows = 0;
self.part_blocks = 0;
self.descriptor_bytes = 0;
self.leaf_bytes = 12;
Ok(())
}
async fn complete(&mut self) -> Result<ArtifactMetadata> {
require(!self.failed, "sample writer failed previously")?;
self.flush_block().await?;
self.finish_part().await?;
let bytes = serde_json::to_vec(&self.manifest).context(SerdeJsonSnafu)?;
require(
bytes.len() <= MANIFEST_LIMIT,
"samples manifest exceeds limit",
)?;
let mut file_size = bytes.len() as u64;
let mut objects = vec![ArtifactObject {
path: self.manifest.path(),
file_size,
}];
for part in &self.manifest.files {
objects.push(ArtifactObject {
path: self.manifest.part_path(part.part_id),
file_size: part.file_size,
});
file_size = file_size
.checked_add(part.file_size)
.ok_or_else(|| invalid("sample artifact size overflow"))?;
}
self.outputs.open(self.manifest.path()).await?;
self.outputs.write(bytes).await?;
self.outputs.close().await?;
self.outputs.committed = true;
Ok(ArtifactMetadata {
format: ArtifactFormat::ColumnarSamples,
generation: self.manifest.generation,
objects,
file_size,
})
}
}
#[async_trait]
impl FormatWriter for SamplesWriter {
async fn write(&mut self, batch: &RecordBatch) -> Result<()> {
require(!self.failed, "sample writer failed previously")?;
self.failed = true;
let result = self.write_batch(batch).await;
if result.is_ok() {
self.failed = false;
}
result
}
async fn finish(mut self: Box<Self>) -> Result<ArtifactMetadata> {
let result = self.complete().await;
if result.is_err()
&& let Err(error) = self.outputs.cleanup().await
{
common_telemetry::warn!(error; "Cleaning failed samples build");
}
result
}
async fn abort(mut self: Box<Self>) -> Result<()> {
self.outputs.cleanup().await
}
}
fn encode_block(rows: &[Row], manifest: &Manifest, ordinal: u64) -> Result<(Block, Vec<Vec<u8>>)> {
let first = rows.first().ok_or_else(|| invalid("empty sample block"))?;
let last = rows.last().ok_or_else(|| invalid("empty sample block"))?;
let mut block = Block {
id: first.id,
min: first.time,
max: last.time,
ordinal,
rows: rows.len() as u32,
columns: Vec::new(),
};
let mut chunks = Vec::new();
let mut push = |role, id, encoding, bytes: Vec<u8>, nulls| -> Result<()> {
let (payload, bytes) = pack(&bytes, 0, encoding, false)?;
block.columns.push(Column {
role,
id,
payload,
nulls,
});
chunks.push(bytes);
Ok(())
};
let (encoding, bytes) = codec::encode_time(&rows.iter().map(|r| r.time).collect::<Vec<_>>());
push(1, manifest.timestamp.column_id, encoding, bytes, 0)?;
let (encoding, bytes) =
codec::encode_sequence(&rows.iter().map(|r| r.sequence).collect::<Vec<_>>());
push(2, 0, encoding, bytes, 0)?;
let (encoding, bytes) =
codec::encode_operations(&rows.iter().map(|r| r.operation).collect::<Vec<_>>());
push(3, 0, encoding, bytes, 0)?;
for (index, field) in manifest.fields.iter().enumerate() {
let values: Vec<_> = rows.iter().map(|r| r.fields[index]).collect();
let nulls = values.iter().filter(|v| v.is_none()).count();
if nulls != rows.len() {
push(
4,
field.column_id,
5,
codec::encode_float(&values),
nulls as u32,
)?;
}
}
Ok((block, chunks))
}
src/mito2/src/series_index/tests.rs
+29 -4
View File
@@ -67,11 +67,25 @@ pub(crate) async fn prepare_region_with_config(
env: &mut TestEnv,
timestamps: &[i64],
config: MitoConfig,
) -> (MitoEngine, MitoRegionRef) {
prepare_region_with_metadata(
env,
timestamps,
config,
Arc::new(sst_region_metadata_with_encoding(
PrimaryKeyEncoding::Sparse,
)),
)
.await
}
pub(crate) async fn prepare_region_with_metadata(
env: &mut TestEnv,
timestamps: &[i64],
config: MitoConfig,
metadata: store_api::metadata::RegionMetadataRef,
) -> (MitoEngine, MitoRegionRef) {
let engine = env.create_engine(config).await;
let metadata = Arc::new(sst_region_metadata_with_encoding(
PrimaryKeyEncoding::Sparse,
));
let region_id = RegionId::new(1, 1);
let mut request = CreateRequestBuilder::new().build();
request.column_metadatas = metadata.column_metadatas.clone();
@@ -129,7 +143,18 @@ pub(crate) async fn prepare_region_with_config(
0,
)),
ValueData::TimestampMillisecondValue(ts),
ValueData::U64Value(1),
if metadata
.field_columns()
.next()
.unwrap()
.column_schema
.data_type
== datatypes::prelude::ConcreteDataType::float64_datatype()
{
ValueData::F64Value(1.0)
} else {
ValueData::U64Value(1)
},
])],
},
hint: Some(WriteHint {
src/promql/src/histogram_eval.rs
+31 -20
View File
@@ -97,7 +97,17 @@ pub fn evaluate_histograms(
schema: &Arc<SeriesSchema>,
series: &[Series],
) -> Result<Vec<Series>> {
let mut groups: HashMap<LabelSet, Group<'_>> = HashMap::new();
// The groups in the order their first member arrived, so the result is in one order on every
// run, as `merge_same_labelset` keeps it.
let mut index: HashMap<LabelSet, usize> = HashMap::new();
let mut groups: Vec<Group<'_>> = Vec::new();
let mut group_of = |metric: MetricName| {
*index.entry(label_set(&metric)).or_insert_with(|| {
groups.push(Group::new(metric));
groups.len() - 1
})
};
let mut members: Vec<(usize, Option<f64>, &Series)> = Vec::new();
for series in series {
grid_axis(series)?;
if let Some(upper_bound) = series
@@ -107,26 +117,27 @@ pub fn evaluate_histograms(
{
let mut metric = series.metric.clone();
metric.remove(BUCKET_LABEL);
groups
.entry(label_set(&metric))
.or_insert_with(|| Group::new(metric))
.classic
.push((upper_bound, series));
members.push((group_of(metric), Some(upper_bound), series));
}
if has_histograms(series) {
let metric = series.metric.clone();
let group = groups
.entry(label_set(&metric))
.or_insert_with(|| Group::new(metric));
// A label set carries at most one native histogram series: a second one would
// overwrite the first and drop it from the result.
debug_assert!(
group.native.is_none(),
"one label set carries at most one native histogram series; a second one for \
the same label set means the input handed one label set over as several \
series, which a gap in the axis is there to avoid"
);
group.native = Some(series);
members.push((group_of(series.metric.clone()), None, series));
}
}
for (slot, upper_bound, series) in members {
let group = &mut groups[slot];
match upper_bound {
Some(upper_bound) => group.classic.push((upper_bound, series)),
None => {
// A label set carries at most one native histogram series: a second one would
// overwrite the first and drop it from the result.
debug_assert!(
group.native.is_none(),
"one label set carries at most one native histogram series; a second one for \
the same label set means the input handed one label set over as several \
series, which a gap in the axis is there to avoid"
);
group.native = Some(series);
}
}
}
@@ -135,7 +146,7 @@ pub fn evaluate_histograms(
let mut classic_buckets = Vec::new();
let mut native_buckets = Vec::new();
let mut values = vec![None; schema.fields.len()];
for mut group in groups.into_values() {
for mut group in groups {
// One builder serves every group, so each result series starts explicitly.
builder.begin_series();
// The positions any member carries, merged once: the holes no member holds are never
src/promql/src/map.rs
+4
View File
@@ -43,6 +43,7 @@ use datatypes::series::time::{GridRange, TimeAxis};
use crate::functions::Round;
use crate::grid_series::GridSeriesBuilder;
use crate::map_time::DateFunction;
/// Float math supported by the evaluated-series kernel.
#[derive(Clone, Copy, Debug)]
@@ -88,6 +89,8 @@ pub enum MathOp {
ClampMax {
max: f64,
},
/// A date function of the value read as Unix seconds, such as `hour(v)`.
Date(DateFunction),
}
/// Float arithmetic between a vector and a scalar, as in Prometheus.
@@ -301,6 +304,7 @@ impl MathOp {
Self::Floor => v.floor(),
Self::Sqrt => v.sqrt(),
Self::Round { nearest } => Round::round_to_nearest(v, nearest),
Self::Date(function) => function.of_seconds(v),
}
}
src/promql/src/map_time.rs
+22 -87
View File
@@ -15,17 +15,13 @@
//! `time()` and the date functions over evaluated series, following Prometheus `funcTime` and
//! `dateWrapper`.
use std::sync::Arc;
use datafusion::arrow::array::{Array, Float64Array};
use datafusion::arrow::array::Float64Array;
use datafusion::error::{DataFusionError, Result};
use datatypes::series::MetricName;
use datatypes::series::samples::{
DenseSamplesRef, SampleRef, Samples, Series, SeriesFields, SeriesSchema,
};
use datatypes::series::samples::{Samples, Series, SeriesFields, SeriesSchema};
use datatypes::series::time::{GridRange, TimeAxis};
use crate::grid_series::{GridSeriesBuilder, METADATA_LABELS, external};
use crate::grid_series::external;
#[cfg(test)]
mod tests;
@@ -86,86 +82,6 @@ pub fn date_of_eval_time(function: DateFunction, schema: &SeriesSchema) -> Resul
eval_time_series(schema, |ms| function.apply(ms / 1000))
}
/// A date function over `series`: each float value is read as Unix seconds and replaced by the
/// date component, and the metric name is dropped. Missing values stay missing; histogram
/// samples are dropped, so the result is `None` when no float remains.
///
/// A missing value is a hole like any other, but a hole only skips a position: the result is one
/// series over the runs of consecutive positions that hold a value.
///
/// Series that share a label set once their names are dropped must then go through
/// [`merge_same_labelset`](crate::grid_series::merge_same_labelset).
pub fn map_date(
function: DateFunction,
schema: &Arc<SeriesSchema>,
series: &Series,
) -> Result<Option<Series>> {
if series.time.grid().is_none() {
return Err(DataFusionError::Execution(
"date functions require an evaluated series".into(),
));
}
let mut metric = series.metric.clone();
metric.remove_all(&METADATA_LABELS);
let map = |seconds: f64| function.apply(go_int64(seconds));
// The fast path maps every float in place, so it needs each field to be entirely present: it
// keeps the input axis as it is. A hole means the mapped rows are fewer than the axis rows,
// so it falls through to the builder, which rebuilds the axis from the rows that survive.
let all_floats = series.fields.iter().all(|samples| {
samples.segments().iter().all(|segment| {
matches!(segment.dense(), Some(DenseSamplesRef::Float(floats)) if floats.null_count() == 0)
})
});
if all_floats {
let fields = series
.fields
.iter()
.map(|samples| {
let segments = samples
.segments()
.iter()
.map(|segment| {
let Some(DenseSamplesRef::Float(floats)) = segment.dense() else {
unreachable!("checked above");
};
Samples::from_float(floats.unary(map))
})
.collect();
Samples::try_from_chunks(segments).map_err(external)
})
.collect::<Result<Vec<_>>>()?;
let fields = SeriesFields::try_from_vec(fields).map_err(external)?;
let series =
Series::try_new(schema, metric, series.time.clone(), fields).map_err(external)?;
return Ok(Some(series));
}
// Holes in any field drop rows, so the builder names the positions that survive.
let mut builder = GridSeriesBuilder::try_new(schema.clone())?;
let mut values = vec![None; series.fields.len()];
// The axis names the positions to evaluate and a cursor per field reads them in order, so an
// axis of several runs costs its rows and not a scan per row.
let mut cursors = series
.fields
.iter()
.map(|samples| samples.cursor())
.collect::<Vec<_>>();
for (position, row) in series.time.positions() {
for (value, samples) in values.iter_mut().zip(&mut cursors) {
*value = match samples
.at(row)
.expect("the axis and the fields are aligned")
{
SampleRef::Float(seconds) => Some(map(seconds)),
SampleRef::Missing | SampleRef::Histogram(_) => None,
};
}
builder.push(position, &values)?;
}
builder.finish(metric)
}
fn eval_time_series(schema: &SeriesSchema, value: impl Fn(i64) -> f64) -> Result<Series> {
let grid = schema.eval_grid.clone().ok_or_else(|| {
DataFusionError::Execution("time functions require an evaluation grid".into())
@@ -204,6 +120,25 @@ const UNIX_TO_ABSOLUTE: i64 = (1969 * 365 + 1969 / 4 - 1969 / 100 + 1969 / 400)
+ (ABSOLUTE_YEARS as i64 / 400 * 146_097 + MARCH_THRU_DECEMBER as i64) * 86_400;
impl DateFunction {
/// The PromQL name of the function.
pub(crate) fn name(self) -> &'static str {
match self {
Self::DayOfMonth => "day_of_month",
Self::DayOfWeek => "day_of_week",
Self::DayOfYear => "day_of_year",
Self::DaysInMonth => "days_in_month",
Self::Hour => "hour",
Self::Minute => "minute",
Self::Month => "month",
Self::Year => "year",
}
}
/// The component of `seconds` read as a Unix time, as Prometheus truncates it with `int64`.
pub(crate) fn of_seconds(self, seconds: f64) -> f64 {
self.apply(go_int64(seconds))
}
/// The component of Unix second `seconds` in UTC, computed the way Go's `time` package does,
/// including its wrapping arithmetic for instants out of its range.
fn apply(self, seconds: i64) -> f64 {
src/promql/src/map_time/tests.rs
+27 -22
View File
@@ -24,12 +24,13 @@ use datatypes::series::samples::{
};
use datatypes::series::time::{EvalGrid, GridRange, GridSpan, GridSpec, TimeAxis};
use super::{DateFunction, QueryContextFunction, date_of_eval_time, map_date, time};
use super::{DateFunction, QueryContextFunction, date_of_eval_time, time};
use crate::grid_series::merge_same_labelset;
use crate::grid_series::reference::{
Labels, Points, canonical_bits, golden_cases, labels_of, metric_of, points_by_labels,
points_of, run_ranges, series_of, sparse_series,
};
use crate::map::{MathOp, map_math};
fn schema(grid: Arc<EvalGrid>) -> Arc<SeriesSchema> {
Arc::new(SeriesSchema {
@@ -71,7 +72,8 @@ fn time_functions_match_prometheus() {
.iter()
.filter_map(|(labels, points)| series_of(&schema, labels, points))
{
actual.extend(map_date(function(name), &schema, &series).unwrap());
actual
.extend(map_math(&schema, &series, MathOp::Date(function(name))).unwrap());
}
actual
}
@@ -107,7 +109,7 @@ fn time_functions_match_prometheus() {
/// Histogram samples are dropped and the series trimmed to the remaining floats; chunked floats
/// are mapped chunk by chunk.
#[test]
fn map_date_drops_histograms_and_maps_chunks() {
fn date_functions_drop_histograms_and_map_chunks() {
let grid = Arc::new(EvalGrid::try_new(GridSpec::try_new(0, 1000, 4).unwrap()).unwrap());
let schema = schema(grid.clone());
let range = GridRange::try_new(grid, 1, 3).unwrap();
@@ -155,7 +157,7 @@ fn map_date_drops_histograms_and_maps_chunks() {
Samples::try_from_histogram(UInt32Array::from(vec![0]), histograms.finish().unwrap())
.unwrap();
let mixed = Samples::try_from_chunks(vec![histogram, floats(&[3600.0, 7200.0])]).unwrap();
let hours = map_date(DateFunction::Hour, &schema, &series(mixed))
let hours = map_math(&schema, &series(mixed), MathOp::Date(DateFunction::Hour))
.unwrap()
.expect("a mixed series with one float");
assert_eq!(
@@ -168,9 +170,13 @@ fn map_date_drops_histograms_and_maps_chunks() {
);
let chunked = Samples::try_from_chunks(vec![floats(&[60.0]), floats(&[120.0, 180.0])]).unwrap();
let minutes = map_date(DateFunction::Minute, &schema, &series(chunked))
.unwrap()
.expect("a chunked float series");
let minutes = map_math(
&schema,
&series(chunked),
MathOp::Date(DateFunction::Minute),
)
.unwrap()
.expect("a chunked float series");
assert_eq!(
points_of(&minutes),
BTreeMap::from([
@@ -181,13 +187,13 @@ fn map_date_drops_histograms_and_maps_chunks() {
);
}
/// `map_date` reads a series of several runs by the positions its axis carries.
/// A date function reads a series of several runs by the positions its axis carries.
///
/// A sparse input cannot be mapped through the in-place fast path when a field holds a hole, so
/// the piece-cutting path runs: the result must report the right positions, not an offset from
/// the first one.
#[test]
fn map_date_reads_a_sparse_axis_by_its_positions() {
fn date_functions_read_a_sparse_axis_by_its_positions() {
let grid = Arc::new(EvalGrid::try_new(GridSpec::try_new(0, 1000, 8).unwrap()).unwrap());
let schema = schema(grid.clone());
// Positions 1, 2 and 6 hold seconds; the axis skips 3, 4 and 5.
@@ -205,7 +211,7 @@ fn map_date_reads_a_sparse_axis_by_its_positions() {
assert_eq!(input.time.first_position(), Some(1));
assert_eq!(input.time.last_position(), Some(6));
let output = map_date(DateFunction::Minute, &schema, &input)
let output = map_math(&schema, &input, MathOp::Date(DateFunction::Minute))
.unwrap()
.expect("three rows");
// No field holds a hole, so the fast path keeps the axis as it is: one series, three rows.
@@ -227,8 +233,8 @@ fn map_date_reads_a_sparse_axis_by_its_positions() {
// `minute()` of Unix seconds 1000, 2000 and 6000.
assert_eq!(read, vec![(1, 16.0), (2, 33.0), (6, 40.0)]);
// A hole in the field keeps the fast path from applying, so the axis is rebuilt from the rows
// that survive: still one series, now over the runs of consecutive valued positions.
// A hole in a float field stays a missing row on the input's axis, as with every map, and the
// positions around it are mapped.
let mut holed = Points::new();
for (position, seconds) in [(2, 120), (3, 180), (4, 240), (6, 360), (7, 420)] {
holed.insert(position, f64::to_bits(seconds as f64));
@@ -239,10 +245,10 @@ fn map_date_reads_a_sparse_axis_by_its_positions() {
&holed,
)
.unwrap();
let output = map_date(DateFunction::Minute, &schema, &input)
let output = map_math(&schema, &input, MathOp::Date(DateFunction::Minute))
.unwrap()
.expect("five rows");
assert_eq!(run_ranges(&output), vec![(2, 3), (6, 2)]);
assert_eq!(run_ranges(&output), run_ranges(&input));
assert_eq!(
points_of(&output),
BTreeMap::from([
@@ -254,10 +260,10 @@ fn map_date_reads_a_sparse_axis_by_its_positions() {
])
);
// A sparse axis *and* a hole in the field, with the samples split over two chunks: the fast
// path cannot apply, so the piece-cutting path must walk the axis' positions and rebuild the
// axis from the rows that survive. Positions 0, 1, 3, 4 and 5 hold rows and position 3 is the
// hole, so a walk that counted rows from the first position would map the wrong seconds.
// A sparse axis *and* a hole in the field, with the samples split over two chunks: each chunk
// is mapped on the input's axis, the hole stays a missing row, and the values land on the
// positions the axis names. Positions 0, 1, 3, 4 and 5 hold rows and position 3 is the hole,
// so a walk that counted rows from the first position would map the wrong seconds.
let time =
TimeAxis::from_grid_spans(grid, vec![GridSpan::new(0, 2), GridSpan::new(3, 3)]).unwrap();
let chunked = Samples::try_from_chunks(vec![
@@ -273,11 +279,10 @@ fn map_date_reads_a_sparse_axis_by_its_positions() {
)
.unwrap();
assert_eq!(input.time.len(), 5);
let output = map_date(DateFunction::Minute, &schema, &input)
let output = map_math(&schema, &input, MathOp::Date(DateFunction::Minute))
.unwrap()
.expect("four rows");
assert_eq!(output.time.len(), 4);
assert_eq!(run_ranges(&output), vec![(0, 2), (4, 2)]);
.expect("five rows");
assert_eq!(run_ranges(&output), run_ranges(&input));
// `minute()` of Unix seconds 60, 120, 240 and 300.
assert_eq!(
points_of(&output),
src/promql/src/native/display.rs
+55 -3
View File
@@ -22,16 +22,18 @@
use std::fmt::{self, Write};
use std::sync::Arc;
use datatypes::series::SeriesOrdering;
use datatypes::series::time::EvalGrid;
use datatypes::series::{MetricName, SeriesOrdering};
use crate::histogram_eval::HistogramFunction;
use crate::map::MathOp;
use crate::native::metrics::{self, NodePath};
use crate::native::plan::{
AggregateKind, AggregateStage, BoundWindow, Grouping, NativeExpr, NativeNode, NativeTree,
NativeType,
AggregateKind, AggregateStage, BoundVectorOp, BoundWindow, Grouping, NativeExpr, NativeNode,
NativeTree, NativeType,
};
use crate::range_eval::RangeFunction;
use crate::select::SortOrder;
use crate::vector_match::{BinaryOp, Cardinality, FillValues, MatchingLabels};
/// The header line of every tree.
@@ -102,6 +104,30 @@ fn describe(node: &NativeNode) -> String {
let ops: Vec<String> = op.ops.iter().map(math_op).collect();
ops.join(", ")
}
NativeExpr::Vector { op, .. } => match op {
BoundVectorOp::TopK { k, top, group } => format!(
"{} {k} {}",
if *top { "topk" } else { "bottomk" },
grouping(group)
),
BoundVectorOp::LimitK { k, group } => format!("limitk {k} {}", grouping(group)),
BoundVectorOp::LimitRatio { ratio } => format!("limit_ratio {ratio}"),
BoundVectorOp::Sort(order) => match order {
SortOrder::Asc => "sort".to_string(),
SortOrder::Desc => "sort_desc".to_string(),
SortOrder::ByLabel(labels) => format!("sort_by_label [{}]", label_list(labels)),
SortOrder::ByLabelDesc(labels) => {
format!("sort_by_label_desc [{}]", label_list(labels))
}
},
BoundVectorOp::Absent { labels } => format!("absent {{{}}}", label_pairs(labels)),
BoundVectorOp::Histogram(HistogramFunction::Quantile(quantile)) => {
format!("histogram_quantile {quantile}")
}
BoundVectorOp::Histogram(HistogramFunction::Fraction { lower, upper }) => {
format!("histogram_fraction {lower} {upper}")
}
},
NativeExpr::Aggregate { op, .. } => format!(
"{} {} [{}] {}",
aggregate_kind(op.kind),
@@ -247,6 +273,7 @@ fn math_op(op: &MathOp) -> String {
MathOp::Clamp { min, max } => format!("clamp {min} {max}"),
MathOp::ClampMin { min } => format!("clamp_min {min}"),
MathOp::ClampMax { max } => format!("clamp_max {max}"),
MathOp::Date(function) => function.name().to_string(),
}
}
@@ -269,6 +296,30 @@ fn binary_op(op: BinaryOp) -> &'static str {
}
}
/// The labels of `metric` as `name="value"` pairs, the metric name first.
fn label_pairs(metric: &MetricName) -> String {
metric
.name()
.map(|name| format!("__name__={name:?}"))
.into_iter()
.chain(
metric
.tags()
.iter()
.map(|tag| format!("{}={:?}", tag.name, tag.value)),
)
.collect::<Vec<_>>()
.join(", ")
}
/// A grouping as `by [labels]` or `without [labels]`.
fn grouping(group: &Grouping) -> String {
match group {
Grouping::By(labels) => format!("by [{}]", label_list(labels)),
Grouping::Without(labels) => format!("without [{}]", label_list(labels)),
}
}
/// What kind of node this is, as its line names it.
fn kind(expr: &NativeExpr) -> &'static str {
match expr {
@@ -278,6 +329,7 @@ fn kind(expr: &NativeExpr) -> &'static str {
NativeExpr::Window { .. } => "Window",
NativeExpr::Map { .. } => "Map",
NativeExpr::Aggregate { .. } => "Aggregate",
NativeExpr::Vector { .. } => "Vector",
NativeExpr::Binary { .. } => "Binary",
}
}
src/promql/src/native/display/tests.rs
+66 -2
View File
@@ -14,16 +14,17 @@
use std::sync::Arc;
use datatypes::series::SeriesOrdering;
use datatypes::series::samples::{FieldSpec, SeriesSchema};
use datatypes::series::time::{EvalGrid, GridSpec};
use datatypes::series::{Label, MetricName, SeriesOrdering};
use super::super::plan::{
AggregateKind, AggregateStage, BoundAggregate, BoundInput, BoundInstant, BoundMap, BoundScalar,
BoundWindow, Grouping, NativeExpr, NativeNode, NativeTree, NativeType,
BoundVectorOp, BoundWindow, Grouping, NativeExpr, NativeNode, NativeTree, NativeType,
};
use crate::map::MathOp;
use crate::range_eval::{RangeFunction, RangeModifier, RangeWindow};
use crate::select::SortOrder;
use crate::vector_match::{BinaryMatching, BinaryOp, Cardinality, FillValues, MatchingLabels};
fn grid(start: i64, step: i64, len: u64) -> Arc<EvalGrid> {
@@ -396,6 +397,69 @@ fn a_without_grouping_does_not_read_as_by() {
);
}
/// A vector operation names its operator, constant and grouping.
#[test]
fn a_vector_operation_names_its_binding() {
let out = grid(0, 60_000, 4);
let window = window_over(
raw_input(),
RangeFunction::Rate,
300_000,
0,
None,
Arc::clone(&out),
);
let lines = [
(
BoundVectorOp::TopK {
k: 3.0,
top: false,
group: Grouping::Without(Arc::new(["job".into()])),
},
" root Vector bottomk 3 without [job]",
),
(
BoundVectorOp::LimitK {
k: 2.0,
group: Grouping::By(Arc::new([])),
},
" root Vector limitk 2 by []",
),
(
BoundVectorOp::LimitRatio { ratio: -0.5 },
" root Vector limit_ratio -0.5",
),
(
BoundVectorOp::Sort(SortOrder::ByLabelDesc(vec!["job".into(), "le".into()])),
" root Vector sort_by_label_desc [job, le]",
),
(
BoundVectorOp::Absent {
labels: MetricName::try_new(
None,
vec![Label {
name: "job".into(),
value: "x".into(),
}],
)
.unwrap(),
},
r#" root Vector absent {job="x"}"#,
),
];
for (op, line) in lines {
let tree = NativeTree::try_new(NativeNode {
expr: NativeExpr::Vector {
input: Box::new(window.clone()),
op,
},
output: evaluated(Arc::clone(&out)),
})
.unwrap();
assert_eq!(tree.to_string().lines().nth(1), Some(line));
}
}
/// A window read with a range modifier names it and the lookback it reads around the window.
#[test]
fn a_modified_window_names_its_modifier() {
src/promql/src/native/execute.rs
+533 -99
View File
@@ -40,7 +40,9 @@
//! operands and runs the matching kernel once the input has ended, because the operation pairs two
//! whole vectors point by point and nothing is decided for a point until both sides are complete.
//! The chunks of one logical series are assembled into the single series the kernel matches before it
//! runs, so a series that arrived in many chunks is one candidate, not one per chunk.
//! runs, so a series that arrived in many chunks is one candidate, not one per chunk. A vector operation
//! (`topk`, `limitk`, `limit_ratio`) reads its one input the same way, because which samples it
//! keeps at a point depends on every series present there.
//!
//! A constant scalar is a value node rather than a stream: it has no input and hands over no chunk,
//! because what it produces is the one value it was bound to and not a series. A binary node with a
@@ -102,6 +104,7 @@ use datatypes::series::{MetricName, SeriesOrdering};
use crate::grid_series::{
GridSample, GridSeriesBuilder, LabelSet, label_set, merge_same_labelset, owned,
};
use crate::histogram_eval::evaluate_histograms;
use crate::instant_eval::select_instants;
use crate::map::{
ArithmeticOp, Comparison, ComparisonOp, MathOp, ScalarOperand, drop_metric_name,
@@ -110,9 +113,11 @@ use crate::map::{
use crate::native::batch::EvaluatedChunk;
use crate::native::metrics::{NodeMetrics, NodePath};
use crate::native::plan::{
BoundInstant, BoundScalar, NativeExpr, NativeNode, NativeTree, NativeType,
BoundInstant, BoundScalar, BoundVectorOp, Grouping, NativeExpr, NativeNode, NativeTree,
NativeType,
};
use crate::range_eval::RangeEvaluator;
use crate::select::{self, SortOrder};
use crate::vector_match::{
BinaryMatching, BinaryOp, Cardinality, FillValues, MatchingLabels,
binary_vectors_with_reservation, validate_binary_operand,
@@ -403,6 +408,8 @@ enum NodeTag {
/// Matches two evaluated vectors and applies one operation to every matched pair of series, or
/// broadcasts one constant over the chunks of one evaluated vector.
Binary,
/// Keeps some samples of its whole input vector at each grid point.
Vector,
}
impl NodeTag {
@@ -415,6 +422,7 @@ impl NodeTag {
Self::Window => "window",
Self::Map => "map",
Self::Binary => "binary",
Self::Vector => "vector operation",
}
}
}
@@ -1268,19 +1276,20 @@ impl InstantRuntime {
}
}
/// One operand of a binary node, chunk by chunk, until it ends.
/// An input a node reads whole before it computes -- one operand of a binary node, or the input of a
/// vector operation -- chunk by chunk, until it ends.
///
/// What is kept is the series each chunk carries, not a concatenation of them: a chunk is retained
/// as it arrives and nothing is copied on the way in, because the chunks of a series can share an
/// allocation and the figure the operand charges for is the data it holds rather than a running copy
/// allocation and the figure the input charges for is the data it holds rather than a running copy
/// of it.
///
/// The kernel matches whole series by label set, so the chunks are assembled into one series per
/// logical series once the input has ended: a series that arrived in a thousand chunks is otherwise
/// a thousand candidates on this side for every candidate of the other, and its own rows would be
/// paired a chunk at a time.
/// The kernels read whole series, so the chunks are assembled into one series per logical series
/// once the input has ended: a series that arrived in a thousand chunks is otherwise a thousand
/// candidates for a binary match or a ranking, and its own rows would be read a chunk at a time.
///
#[derive(Default)]
struct BinaryOperand {
struct RetainedInput {
/// The series whose last chunk did not close it, with the position after its last row.
///
/// `None` when no series is open: nothing has arrived yet, or the last series closed. A label
@@ -1321,7 +1330,7 @@ struct BinaryOperand {
held: usize,
}
impl BinaryOperand {
impl RetainedInput {
/// The bytes `chunk` would add to what this operand holds, read without retaining it.
///
/// It is what [`Self::enqueue`] adds for the same chunk: what the chunk holds on its own, the
@@ -1344,6 +1353,62 @@ impl BinaryOperand {
.saturating_add(end)
}
/// Retains one chunk after checking that it continues what this input handed over.
///
/// An input hands over one series at a time, each chunk of it after the rows the series already
/// reached; a chunk that starts another series while one is open, or goes back inside the open
/// one, is refused before it is retained.
fn retain(&mut self, chunk: EvaluatedChunk, reader: Reader<'_>) -> Result<()> {
// An empty chunk closes a series without holding a row, so it says nothing about where that
// series ended: only a chunk that carries rows can be read as going backwards.
let (first, last) = (
chunk.series.time.first_position(),
chunk.series.time.last_position(),
);
let metric = chunk.series.metric.clone();
// The position a chunk of the open series may start at: the row after the last one that
// series handed over. Nothing is open after a series closed, so the next one starts over.
let end = self.open.as_ref().map_or(0, |(_, end)| *end);
if let Some((open, _)) = &self.open
&& *open != metric
{
let (owner, input) = (reader.owner(), reader.input());
return Err(DataFusionError::Execution(format!(
"{owner} reads whole series, and {input} handed over {metric:?} before {open:?} \
ended"
)));
}
if let Some(first) = first
&& first < end
{
let (owner, input) = (reader.owner(), reader.input());
return Err(DataFusionError::Execution(format!(
"{owner} was handed chunks of {metric:?} by {input} that do not continue each \
other: one starts at position {first}, after the series already reached {end}"
)));
}
self.open = (!chunk.is_last).then(|| (metric, last.map_or(end, |last| last + 1)));
self.enqueue(chunk);
Ok(())
}
/// Refuses an input that ended inside a series.
///
/// An input hands over one series at a time, so the only series that can be open is the last
/// one it handed over: a chunk that did not say `is_last` leaves it open, and nothing came after
/// it to close it.
fn ensure_closed(&self, reader: Reader<'_>) -> Result<()> {
match &self.open {
Some((metric, _)) => Err(DataFusionError::Execution(format!(
"the input of {} ended before the last chunk of a series: {} handed over \
{metric:?} without the chunk that closes it",
reader.owner(),
reader.input()
))),
None => Ok(()),
}
}
/// Retains one chunk, folding the estimate it is charged with into what the operand holds.
fn enqueue(&mut self, chunk: EvaluatedChunk) {
let closing = chunk.is_last;
@@ -1374,7 +1439,7 @@ impl BinaryOperand {
/// The chunk ranges of the logical series this operand closed, in arrival order.
///
/// Read after the input ended: a series that is still open has no end, and
/// [`BinaryRuntime::closed`] refuses such an input before anything is assembled.
/// [`Self::ensure_closed`] refuses such an input before anything is assembled.
fn boundaries(&self) -> impl Iterator<Item = std::ops::Range<usize>> + '_ {
self.ends.iter().scan(0, |start, &end| {
let range = *start..end;
@@ -1451,24 +1516,25 @@ impl BinaryOperand {
/// the kernel matches and reads, so it must be on the schema's grid and hold as many rows as its
/// axis places, which is what [`Series::try_new`] checks. What it allocates is
/// [`Self::assembly_bytes`], which a caller reserves before this runs.
fn assemble(&self, schema: &SeriesSchema, path: &NodePath, side: usize) -> Result<Vec<Series>> {
fn assemble(&self, schema: &SeriesSchema, reader: Reader<'_>) -> Result<Vec<Series>> {
let grid = schema.eval_grid.clone().ok_or_else(|| {
let (owner, input) = (reader.owner(), reader.input());
DataFusionError::Execution(format!(
"the binary node at {path} assembles its {} operand on an evaluation grid, and the \
operand's schema names none",
side_name(side)
"{owner} assembles {input} on an evaluation grid, and its schema names none"
))
})?;
let mut assembled = Vec::with_capacity(self.ends.len());
for boundary in self.boundaries() {
let chunks = &self.series[boundary];
let metric = chunks.first().map(|chunk| chunk.metric.clone()).ok_or_else(|| {
DataFusionError::Execution(format!(
"the binary node at {path} closed a logical series of its {} operand without a \
chunk to assemble",
side_name(side)
))
})?;
let metric = chunks
.first()
.map(|chunk| chunk.metric.clone())
.ok_or_else(|| {
let (owner, input) = (reader.owner(), reader.input());
DataFusionError::Execution(format!(
"{owner} closed a logical series of {input} without a chunk to assemble"
))
})?;
let runs = chunks
.iter()
.map(|chunk| chunk.time.runs().count())
@@ -1484,10 +1550,10 @@ impl BinaryOperand {
let mut segments = Vec::with_capacity(chunks.len());
for chunk in chunks {
let field = chunk.fields.get(index).ok_or_else(|| {
let (owner, input) = (reader.owner(), reader.input());
DataFusionError::Execution(format!(
"the binary node at {path} cannot assemble its {} operand: a chunk of \
it carries {} fields, not {}",
side_name(side),
"{owner} cannot assemble {input}: a chunk of it carries {} fields, not \
{}",
chunk.fields.len(),
schema.fields.len()
))
@@ -1532,7 +1598,7 @@ struct BinaryRuntime {
/// operand and `1` the right one, the order the tree declares them in.
schemas: [Arc<SeriesSchema>; 2],
/// The retained chunks of each operand, in the same order.
operands: [BinaryOperand; 2],
operands: [RetainedInput; 2],
/// Whether the kernel has run. It runs at most once: the operands it read are gone afterwards.
computed: bool,
}
@@ -1545,13 +1611,38 @@ fn side_name(index: usize) -> &'static str {
}
}
/// The node a [`RetainedInput`] belongs to, which its messages name; formatted only when one is.
#[derive(Clone, Copy)]
enum Reader<'a> {
/// The operand at `side` of the binary node at `path`.
Operand { path: &'a NodePath, side: usize },
/// The input of the vector operation at the path.
Vector(&'a NodePath),
}
impl Reader<'_> {
fn owner(self) -> String {
match self {
Self::Operand { path, .. } => format!("the binary node at {path}"),
Self::Vector(path) => format!("the vector operation at {path}"),
}
}
fn input(self) -> String {
match self {
Self::Operand { side, .. } => format!("its {} operand", side_name(side)),
Self::Vector(_) => "its input".to_string(),
}
}
}
impl BinaryRuntime {
/// The run-time state of the binary operation `matching` over its two operands.
fn new(matching: Arc<BinaryMatching>, schemas: [Arc<SeriesSchema>; 2]) -> Self {
Self {
matching,
schemas,
operands: [BinaryOperand::default(), BinaryOperand::default()],
operands: [RetainedInput::default(), RetainedInput::default()],
computed: false,
}
}
@@ -1577,8 +1668,8 @@ impl BinaryRuntime {
let [left, right] = &self.operands;
let [left_schema, right_schema] = &self.schemas;
Ok([
left.assemble(left_schema, path, 0)?,
right.assemble(right_schema, path, 1)?,
left.assemble(left_schema, Reader::Operand { path, side: 0 })?,
right.assemble(right_schema, Reader::Operand { path, side: 1 })?,
])
}
@@ -1596,20 +1687,11 @@ impl BinaryRuntime {
/// Reports an operand that ended inside a series.
///
/// A side hands over one series at a time, so the only series that can be open is the last one
/// it handed over: a chunk that did not say `is_last` leaves it open, and nothing came after it
/// to close it. The kernel is not run over such an operand: it would answer the pairs of a
/// series whose remaining rows were never read.
/// The kernel is not run over such an operand: it would answer the pairs of a series whose
/// remaining rows were never read.
fn closed(&self, path: &NodePath) -> Result<()> {
for (side, operand) in self.operands.iter().enumerate() {
let Some((metric, _)) = &operand.open else {
continue;
};
return Err(DataFusionError::Execution(format!(
"the input of the binary node at {path} ended before the last chunk of a series: \
its {} operand handed over {metric:?} without the chunk that closes it",
side_name(side)
)));
operand.ensure_closed(Reader::Operand { path, side })?;
}
Ok(())
}
@@ -1632,41 +1714,10 @@ impl BinaryRuntime {
))
},
)?;
// An empty chunk closes a series without holding a row, so it says nothing about where that
// series ended: only a chunk that carries rows can be read as going backwards.
let (first, last) = (
chunk.series.time.first_position(),
chunk.series.time.last_position(),
);
let metric = chunk.series.metric.clone();
let Some(operand) = self.operands.get_mut(side) else {
return Err(no_binary(path));
};
// The position a chunk of the open series may start at: the row after the last one that
// series handed over. Nothing is open after a series closed, so the next one starts over.
let end = operand.open.as_ref().map_or(0, |(_, end)| *end);
if let Some((open, _)) = &operand.open
&& *open != metric
{
return Err(DataFusionError::Execution(format!(
"the binary node at {path} combines whole series, and its {} operand handed over \
{metric:?} before {open:?} ended",
side_name(side)
)));
}
if let Some(first) = first
&& first < end
{
return Err(DataFusionError::Execution(format!(
"the chunks of {metric:?} handed to the {} operand of the binary node at {path} do \
not continue each other: one starts at position {first}, after the series already \
reached {end}",
side_name(side)
)));
}
operand.open = (!chunk.is_last).then(|| (metric, last.map_or(end, |last| last + 1)));
operand.enqueue(chunk);
Ok(())
operand.retain(chunk, Reader::Operand { path, side })
}
/// Runs the kernel over both operands and returns the series it produced.
@@ -1750,6 +1801,215 @@ impl BinaryRuntime {
}
}
/// What a vector node holds: its whole input until it ends.
///
/// Which samples `topk` or `limitk` keeps at a point depends on every series present there, so,
/// like a binary node, this node retains every chunk its input hands over, assembles the chunks of
/// each logical series, and runs its kernel once, after the input has ended. The results are queued
/// in the node's own output queue and handed over from there.
struct VectorRuntime {
op: BoundVectorOp,
/// The layout of the input, which is also what the node produces.
schema: Arc<SeriesSchema>,
input: RetainedInput,
/// Whether the input's pieces of one label set are merged before the kernel reads them, which
/// `limitk` needs when its input can hold them.
merge_input: bool,
/// Whether the kernel has run. It runs at most once: the input it read is gone afterwards.
computed: bool,
}
impl VectorRuntime {
fn new(op: BoundVectorOp, schema: Arc<SeriesSchema>) -> Self {
Self {
op,
schema,
input: RetainedInput::default(),
merge_input: false,
computed: false,
}
}
/// Runs the kernel over the whole input and returns the series it kept, or `None` when it
/// already ran.
///
/// The reservation covers the retained chunks, the assembled series and a rough estimate of the kernel's workspace and of the series it rebuilds
/// ([`vector_workspace_bytes`]). The kernels do not report their workspace as they allocate,
/// so the estimate is reserved once, before the kernel runs.
fn compute(&mut self, path: &NodePath, buffer: &mut NodeBuffer) -> Result<Option<Vec<Series>>> {
if self.computed {
return Ok(None);
}
self.input.ensure_closed(Reader::Vector(path))?;
let retained = self.input.held;
let assembly = self
.input
.assembly_bytes(&self.schema)
.ok_or_else(|| vector_overflow(path))?;
buffer.grow(assembly)?;
let assembled = match self.input.assemble(&self.schema, Reader::Vector(path)) {
Ok(assembled) => assembled,
Err(error) => {
buffer.resize(retained)?;
return Err(error);
}
};
// `limitk` keeps the first series in the order its input hands them over, as Prometheus
// takes the order of the vector it is given -- a sort below it decides that order. Pieces
// of one label set are one series there, placed where the label set first arrived, so
// they are merged first; ranked apart, a later piece would be counted after the series
// that arrived between them.
let assembled = if self.merge_input {
let rows = assembled.iter().map(|series| series.time.len()).sum();
buffer.grow(merge_workspace_bytes(
rows,
assembled.len(),
self.schema.fields.len(),
))?;
merge_same_labelset(&self.schema, assembled)?
} else {
assembled
};
buffer.grow(vector_workspace_bytes(&self.op, &self.schema, &assembled))?;
let metrics = Arc::clone(&buffer.metrics);
let output = {
let _timer = metrics.timer();
vector_series(&self.op, &self.schema, assembled)?
};
self.input.release();
self.computed = true;
Ok(Some(output))
}
}
/// The error for a vector operation whose reservation does not fit an address space.
fn vector_overflow(path: &NodePath) -> DataFusionError {
DataFusionError::Execution(format!(
"the input of the vector operation at {path} does not fit one address space: the bytes it would \
reserve overflow"
))
}
/// Runs the kernel `op` binds over the whole input.
fn vector_series(
op: &BoundVectorOp,
schema: &Arc<SeriesSchema>,
series: Vec<Series>,
) -> Result<Vec<Series>> {
let grouping = |group: &Grouping| match group {
Grouping::By(labels) => select::Grouping::By(labels.to_vec()),
Grouping::Without(labels) => select::Grouping::Without(labels.to_vec()),
};
match op {
BoundVectorOp::TopK { k, top, group } => {
select::topk(*k, *top, &grouping(group), schema, &series)
}
BoundVectorOp::LimitK { k, group } => select::limitk(*k, &grouping(group), schema, &series),
BoundVectorOp::LimitRatio { ratio } => select::limit_ratio(*ratio, schema, &series),
// A sort hands back the series it was given, rebuilding only those `sort` and `sort_desc`
// drop histogram samples from.
BoundVectorOp::Sort(order) => select::sort_instant(order, schema, series),
// The kernel groups bucket series by label set and takes one native histogram series per
// set, so pieces of one label set -- an empty tag and an absent one, or what a map folded
// -- are merged first, and two of them holding one point fail as they do elsewhere.
BoundVectorOp::Histogram(function) => {
evaluate_histograms(*function, schema, &merge_same_labelset(schema, series)?)
}
BoundVectorOp::Absent { labels } => Ok(select::absent(schema, &series, labels.clone())?
.into_iter()
.collect()),
}
}
/// A rough estimate of what the vector operation `op` allocates over `series`, beside the input it reads.
/// Like [`merge_workspace_bytes`] it is not an allocation bound: vector slack and the hash table's
/// own layout are not measured.
///
/// - `topk`, `bottomk`, `limitk` and `limit_ratio`, per row: the `(point, row)` a kept sample is
/// recorded as, the candidate it is ranked as and the value each field rebuilds it into; per
/// series: its place in the label order and its group, its label pairs in the group key, its
/// cursor and the series and field descriptors of its result; and the histograms `limitk` and
/// `limit_ratio` copy into their results, at the widest histogram of each field.
/// - A sort: the value each series is keyed by, and the floats `sort` and `sort_desc` rebuild a
/// series into when it holds histograms; the label sorts move the series they were given.
/// - `absent`: a presence flag and a result value per grid point and field.
/// - `histogram_quantile` and `histogram_fraction`: the merge of label-set pieces before the kernel
/// ([`merge_workspace_bytes`]); per series, its group entry and label pairs and a cursor per
/// field; per row, the bucket it contributes and a result value per field.
fn vector_workspace_bytes(op: &BoundVectorOp, schema: &SeriesSchema, series: &[Series]) -> usize {
let fields = schema.fields.len();
match op {
BoundVectorOp::Sort(order) => {
let rebuilds = matches!(order, SortOrder::Asc | SortOrder::Desc);
return series.iter().fold(0usize, |total, item| {
let floats = if rebuilds
&& item
.fields
.iter()
.any(|field| histogram_row_bytes(field).is_some())
{
item.time.len().saturating_mul(std::mem::size_of::<f64>())
} else {
0
};
total
.saturating_add(std::mem::size_of::<(f64, Series)>())
.saturating_add(floats)
});
}
BoundVectorOp::Absent { .. } => {
let points = schema.eval_grid.as_ref().map_or(0, |grid| {
usize::try_from(grid.spec().len).unwrap_or(usize::MAX)
});
return points.saturating_mul(fields.saturating_mul(1 + std::mem::size_of::<f64>()));
}
BoundVectorOp::Histogram(_) => {
let rows: usize = series.iter().map(|item| item.time.len()).sum();
let label = std::mem::size_of::<(Arc<str>, Arc<str>)>();
let row = std::mem::size_of::<(f64, f64)>() + fields * std::mem::size_of::<f64>();
let per_series = std::mem::size_of::<(f64, &Series)>()
+ fields * std::mem::size_of::<RowCursor<'_>>();
return series.iter().fold(
merge_workspace_bytes(rows, series.len(), fields),
|total, item| {
total
.saturating_add(per_series)
.saturating_add((item.metric.tags().len() + 1).saturating_mul(label))
.saturating_add(item.time.len().saturating_mul(row))
},
);
}
BoundVectorOp::TopK { .. }
| BoundVectorOp::LimitK { .. }
| BoundVectorOp::LimitRatio { .. } => {}
}
let row = std::mem::size_of::<(usize, usize)>()
+ std::mem::size_of::<(usize, usize, f64)>()
+ fields * std::mem::size_of::<f64>();
let per_series = 2 * std::mem::size_of::<usize>()
+ std::mem::size_of::<Vec<(usize, usize)>>()
+ std::mem::size_of::<RowCursor<'_>>()
+ std::mem::size_of::<Series>()
+ fields * std::mem::size_of::<Samples>();
let label = std::mem::size_of::<(Arc<str>, Arc<str>)>();
series.iter().fold(0usize, |total, item| {
let rows = item.time.len();
let labels = item.metric.tags().len() + 1;
let histograms = item
.fields
.iter()
.filter_map(histogram_row_bytes)
.fold(0usize, |bytes, widest| {
bytes.saturating_add(widest.saturating_mul(rows))
});
total
.saturating_add(rows.saturating_mul(row))
.saturating_add(per_series)
.saturating_add(labels.saturating_mul(label))
.saturating_add(histograms)
})
}
/// The operation a scalar broadcast applies, with the side the constant sits on already resolved.
#[derive(Clone, Copy, Debug)]
enum BroadcastOp {
@@ -1978,10 +2238,15 @@ struct NodeRuntime {
/// The chunks this node holds for its consumer, each with the estimate it was queued with.
///
/// An input node's queue is filled by the caller, a window's by the chunks it computed while
/// flushing its collected input, and a binary node's by the results its kernel produced; no node
/// flushing its collected input, and a binary or vector node's by the results its kernel
/// produced; no node
/// uses it for both. A chunk is measured once, as it enters, and the figure it was queued with
/// is given back as it leaves, so neither the queue nor the handover walks it.
queued: VecDeque<(NativeChunk, usize)>,
/// The histogram tables queued results share with each other, charged to the queue as a whole
/// rather than to one result: a table stays alive while any result referencing it is queued,
/// so its charge is given back when the queue empties, not when the first result leaves.
queued_tables: usize,
/// What an instant node holds: the chunks of the series it is selecting, and, while it gathers
/// that series, the workspace the gather reserved for. `None` for every other kind of node.
instant: Option<InstantRuntime>,
@@ -1995,6 +2260,8 @@ struct NodeRuntime {
/// `None` for every other node, and for a binary node that broadcasts a constant instead: that
/// one holds no operand at all and is described by [`NodeRuntime::broadcast`].
binary: Option<BinaryRuntime>,
/// What a vector node holds.
vector: Option<VectorRuntime>,
/// What a binary node holds when one operand is a constant: the operation and the layout of the
/// vector side.
///
@@ -2050,10 +2317,10 @@ impl NodeRuntime {
/// Hands over the next queued chunk, if there is one, giving back what it was queued with.
///
/// It is the one way a chunk leaves this node's queue: an input node hands over what the caller
/// queued, and a binary node hands over what its kernel produced. `values` is how that node
/// counts the values it hands over, because the two do not count the same thing: an input's
/// figure has always been its positions, while a binary result holds one value per field and a
/// field may have none at a position it decided.
/// queued, and a binary or vector node hands over what its kernel produced. `values` is how
/// that node counts the values it hands over, because the two do not count the same thing: an
/// input's figure has always been its positions, while a kernel's result holds one value per
/// field and a field may have none at a position it decided.
fn pop_queued(&mut self, values: fn(&NativeChunk) -> usize) -> Result<Option<NativeChunk>> {
let Some((chunk, bytes)) = self.queued.pop_front() else {
return Ok(None);
@@ -2065,6 +2332,7 @@ impl NodeRuntime {
// hold it.
let _timer = metrics.timer();
self.buffer.shrink(bytes)?;
self.release_queued_tables()?;
}
metrics.record_output(1, values(&chunk));
// An evaluated chunk is addressed on its grid, so its decided positions are grid points; a
@@ -2307,30 +2575,97 @@ impl NodeRuntime {
let Some(output) = output else {
return Ok(());
};
self.queue_results(output, false)
}
/// Retains one chunk of a vector node's input, reserved for before it is retained, as
/// [`Self::retain_binary`] retains an operand's.
fn retain_vector(&mut self, path: &NodePath, chunk: NativeChunk) -> Result<()> {
let NativeChunk::Evaluated(chunk) = chunk else {
return Err(DataFusionError::Execution(format!(
"the vector operation at {path} consumes evaluated chunks, not raw samples"
)));
};
let vector = self.vector.as_mut().ok_or_else(|| no_vector(path))?;
// What a bound tree says the input produces is not proof of what a chunk carries, so the
// chunk is checked against the layout before it is retained.
validate_binary_operand(vector.schema.as_ref(), std::slice::from_ref(&chunk.series))
.map_err(|error| {
DataFusionError::Execution(format!(
"the vector operation at {path} rejected its input: {error}"
))
})?;
let pending = vector.input.pending(&chunk);
let positions = chunk.series.time.len();
self.buffer.grow(pending)?;
let metrics = Arc::clone(&self.buffer.metrics);
{
let _timer = metrics.timer();
let vector = self.vector.as_mut().ok_or_else(|| no_vector(path))?;
vector.input.retain(chunk, Reader::Vector(path))?;
let retained = vector.input.held;
self.buffer.resize(retained)?;
}
metrics.record_input(1, positions);
Ok(())
}
/// Runs this node's vector kernel and queues the series it kept.
fn compute_vector(&mut self, path: &NodePath) -> Result<()> {
let (output, shares_input) = {
let vector = self.vector.as_mut().ok_or_else(|| no_vector(path))?;
let shares_input = matches!(vector.op, BoundVectorOp::Sort(_));
(vector.compute(path, &mut self.buffer)?, shares_input)
};
let Some(output) = output else {
return Ok(());
};
self.queue_results(output, shares_input)
}
/// Queues the series a kernel that reads its whole input produced, and settles the reservation
/// on the queue.
///
/// `shares_input` says the series are the assembled input itself, as a sort hands it back: a
/// field there is one slice per input chunk of buffers and histogram tables the chunks shared,
/// so each series is measured the way the input was ([`operand_chunk_bytes`]) rather than by
/// the allocations its slices belong to, which would count a buffer once per chunk. The tables
/// are billed once, to the queue ([`NodeRuntime::queued_tables`]), since several results can
/// keep one alive.
fn queue_results(&mut self, output: Vec<Series>, shares_input: bool) -> Result<()> {
let mut tables = HashSet::new();
// The results are queued the way a node that produces a chunk per input queues what it
// hands on: the queue is the one source of chunk order and of the bytes queued chunks hold.
let mut closed = 0usize;
for series in output {
// The kernel decided every point of a result series at once, so the one chunk it is
// handed over as closes it.
let shared = shares_input.then(|| {
let (fresh, table_bytes) = unbilled_tables(&series, &tables);
tables.extend(fresh);
self.queued_tables = self.queued_tables.saturating_add(table_bytes);
operand_chunk_bytes(&series)
});
let chunk = NativeChunk::Evaluated(EvaluatedChunk::try_new(series, true)?);
// The result is already admitted: the kernel kept it inside the workspace its callback
// reserved. What the queue keeps is the estimate of the chunk that holds it.
let bytes = chunk.bytes();
// The result is already admitted, within the workspace reserved for the kernel. What the
// queue keeps is the estimate of the chunk that holds it.
let bytes = shared.unwrap_or_else(|| chunk.bytes());
self.queued.push_back((chunk, bytes));
closed += 1;
}
// This node's own accounting of closed series is the results it closed, one per matched
// series: an operand chunk that closed its input series is a closure of the node below, and
// This node's own accounting of closed series is the results it closed, one per result
// series: an input chunk that closed its series is a closure of the node below, and
// counting it here as well would report the same work twice.
self.buffer.metrics.record_completed_series(closed);
// The operands the reservation covered are gone and the queue holds what the kernel
// returned, so the reservation settles on the queue: what a queued result is given back
// with as it leaves is exactly the figure it is held for here.
// The input the reservation covered is gone and the queue holds what the kernel returned,
// so the reservation settles on the queue: what a queued result is given back with as it
// leaves is exactly the figure it is held for here.
let queued = self
.queued
.iter()
.fold(0usize, |total, (_, bytes)| total.saturating_add(*bytes));
.fold(self.queued_tables, |total, (_, bytes)| {
total.saturating_add(*bytes)
});
self.buffer.resize(queued)
}
@@ -2492,9 +2827,19 @@ impl NodeRuntime {
return Ok(None);
};
self.buffer.shrink(bytes)?;
self.release_queued_tables()?;
Ok(Some(chunk))
}
/// Gives back the shared tables' charge once no queued result references them.
fn release_queued_tables(&mut self) -> Result<()> {
if self.queued.is_empty() && self.queued_tables > 0 {
self.buffer
.shrink(std::mem::take(&mut self.queued_tables))?;
}
Ok(())
}
/// Records what this node hands out under its output label set.
///
/// A node that rewrites labels is where two input series become indistinguishable, so this is
@@ -2526,6 +2871,7 @@ impl NodeRuntime {
self.window = None;
self.instant = None;
self.binary = None;
self.vector = None;
self.broadcast = None;
self.source_schema = None;
self.emitted = None;
@@ -2536,6 +2882,7 @@ impl NodeRuntime {
self.open_source_series = false;
self.queued.clear();
self.queued.shrink_to_fit();
self.queued_tables = 0;
// The reservation is returned once, below, for everything this node still held.
self.buffer.release();
}
@@ -2616,6 +2963,7 @@ impl NativeExecutionState {
// two vectors, or a constant broadcast over one -- is decided where the operands'
// layouts are read.
NativeExpr::Binary { .. } => NodeTag::Binary,
NativeExpr::Vector { .. } => NodeTag::Vector,
// A scalar is a value node: it produces the constant it was bound to, hands over no
// chunk and holds no state, so it is built like any other node and pulled like none.
NativeExpr::Scalar(_) => NodeTag::Scalar,
@@ -2629,10 +2977,12 @@ impl NativeExecutionState {
let mut runtime = NodeRuntime {
buffer,
queued: VecDeque::new(),
queued_tables: 0,
instant: None,
window: None,
map: None,
binary: None,
vector: None,
broadcast: None,
source_schema: None,
emitted: None,
@@ -2731,6 +3081,28 @@ impl NativeExecutionState {
// out. A broadcast that keeps the labels it reads is not where two series
// become one either, so it keeps none; the one that drops them does, above.
}
NativeExpr::Vector { op, .. } => {
let schema = node.output.samples().cloned().ok_or_else(|| {
DataFusionError::Execution(format!(
"the vector operation at {path} does not produce samples"
))
})?;
// `topk`, `bottomk`, `limitk` and the value sorts read one value per sample,
// so their kernels take one field; the others keep or test whole samples and
// take any number.
let one_field = matches!(
op,
BoundVectorOp::TopK { .. }
| BoundVectorOp::LimitK { .. }
| BoundVectorOp::Sort(SortOrder::Asc | SortOrder::Desc)
);
if one_field && schema.fields.len() != 1 {
return Err(not_implemented("multi-field Vector", &path));
}
runtime.vector = Some(VectorRuntime::new(op.clone(), schema));
// A vector operation keeps the labels of the series it reads, or answers under
// label sets of its own, so it keeps no ledger of what it handed out.
}
NativeExpr::Scalar(_) | NativeExpr::Aggregate { .. } => {
// A scalar holds nothing: what its consumer reads is the constant the node was
// bound to, which is on the tree rather than in this state. The aggregate is
@@ -2747,7 +3119,8 @@ impl NativeExecutionState {
}
NativeExpr::Instant { input, .. }
| NativeExpr::Window { input, .. }
| NativeExpr::Map { input, .. } => {
| NativeExpr::Map { input, .. }
| NativeExpr::Vector { input, .. } => {
pending.push((input.as_ref(), path.input()));
true
}
@@ -2850,6 +3223,32 @@ impl NativeExecutionState {
None => false,
};
}
NodeTag::Vector => {
// Most vector operations keep samples of the series they read, so pieces of one label
// set stay pieces: they hold disjoint points, so ranking them apart picks
// what ranking the merged series would, and what is above treats them as
// it treats a map's. `absent` answers with one series under the labels it
// was bound to, the histogram functions merge the label sets they answer
// with, and `limitk` counts series in input order, so it merges the pieces
// first; none of them hands over pieces.
let op = match expr {
NativeExpr::Vector { op, .. } => Some(op),
_ => None,
};
let limitk = matches!(op, Some(BoundVectorOp::LimitK { .. }));
if limitk
&& input_holds_shared
&& let Some(vector) = runtimes[index].vector.as_mut()
{
vector.merge_input = true;
}
let whole = limitk
|| matches!(
op,
Some(BoundVectorOp::Absent { .. } | BoundVectorOp::Histogram(_))
);
holds_shared[index] = input_holds_shared && !whole;
}
NodeTag::Window => {
let (drops_metric_name, input_schema) = match expr {
NativeExpr::Window { input, op } => (
@@ -3017,12 +3416,12 @@ impl NativeExecutionState {
/// Reports a stream that ended inside a series, without closing any node.
///
/// Closing is destructive -- a window's evaluator takes itself apart -- and the binary nodes
/// still have to compute after this check, so what is read here is only the state that follows
/// what each node was handed: an input node reports the series it handed over without the chunk
/// that ends it, and a binary node reports an operand that is still open on either side. A
/// truncation is therefore refused before a binary kernel allocates its workspace, rather than
/// after one has already been computed on half a vector.
/// Closing is destructive -- a window's evaluator takes itself apart -- and the binary and
/// vector nodes still have to compute after this check, so what is read here is only the
/// state that follows what each node was handed: an input node reports the series it handed
/// over without the chunk that ends it, and a binary or vector node reports an input that is
/// still open. A truncation is therefore refused before a kernel allocates its workspace, rather
/// than after one has already been computed on half a vector.
fn unclosed(&self) -> Result<()> {
// A source series that never ended is refused before anything is merged: the close a merge
// synthesises would make it look like one that did.
@@ -3033,6 +3432,9 @@ impl NativeExecutionState {
if let Some(binary) = &runtime.binary {
binary.closed(path)?;
}
if let Some(vector) = &runtime.vector {
vector.input.ensure_closed(Reader::Vector(path))?;
}
}
Ok(())
}
@@ -3156,7 +3558,8 @@ impl NativeExecutor {
/// exception: it pairs two whole vectors point by point, so it retains every chunk each operand
/// hands over and reports `None` until [`Self::finish`] ends the input, which is when it
/// computes its result. A binary node that broadcasts a constant is not one: it pairs each
/// value of the vector side with the constant and hands the transformed chunk up as it arrives.
/// value of the vector side with the constant and hands the transformed chunk up as it arrives. A vector node
/// reads its one input whole the same way.
///
/// A chunk's `is_last` says the series whose data it carries ended. It does not say its label
/// set is finished: two source series whose labels were rewritten into one set are two pieces
@@ -3342,6 +3745,15 @@ impl NativeExecutor {
NodeTag::Map | NodeTag::Window | NodeTag::Instant => self.state.nodes[index]
.input
.is_none_or(|child| self.settled(child)),
NodeTag::Vector => {
runtime
.vector
.as_ref()
.is_some_and(|vector| vector.computed)
&& self.state.nodes[index]
.input
.is_none_or(|child| self.settled(child))
}
NodeTag::Binary => {
// A broadcast owes only what its vector side has not handed over yet: it answers
// each chunk as it reads it and keeps nothing, so it is settled exactly when that
@@ -3530,6 +3942,22 @@ impl NativeExecutor {
// does, so its values are counted rather than read as its positions.
runtime.pop_queued(evaluated_values)
}
NodeTag::Vector => {
let child = input.ok_or_else(|| missing_input(tag))?;
// Which samples are kept at a point is not decided until the whole input has
// arrived, so the input is read to the end before anything is handed over.
while let Some(chunk) = self.pull(child)? {
let path = &self.state.paths[index];
self.state.runtimes[index].retain_vector(path, chunk)?;
}
if !self.inputs_finished || !self.settled(child) {
return Ok(None);
}
let path = &self.state.paths[index];
let runtime = &mut self.state.runtimes[index];
runtime.compute_vector(path)?;
runtime.pop_queued(evaluated_values)
}
}
}
@@ -3692,9 +4120,10 @@ impl NativeExecutor {
/// Maps one chunk with `ops`, keeping its axis and its closing.
///
/// Mapping is point-wise over valued positions, so a chunk whose values survive keeps exactly the
/// rows it came with: a position no field fills is not a row of any chunk, and mapping does not
/// invent one. `None` means the operation emptied every position, which yields no chunk at all
/// Mapping is point-wise over valued positions: a float chunk keeps the rows it came with, and a
/// chunk holding native histograms loses the rows only histograms filled, as math skips them; a
/// position no field fills is not a row of any chunk, and mapping does not invent one. `None` means
/// the operation emptied every position, which yields no chunk at all
/// unless this chunk closed its series: a closing chunk is still handed over, holding no position,
/// because a consumer tells "nothing more here" from "no more input" by that chunk alone.
fn map_chunk(
@@ -3706,7 +4135,8 @@ fn map_chunk(
for &op in ops {
series = match map_math(schema, &series, op)? {
Some(mapped) => mapped,
// `clamp` with `max < min` is the empty vector: every position it held was decided and
// `clamp` with `max < min` is the empty vector, and math skips native histograms, so a
// series without a float value has none left: every position it held was decided and
// holds no value now, so there is no row to hand over.
None => {
return if chunk.is_last {
@@ -3868,6 +4298,10 @@ fn no_binary(path: &NodePath) -> DataFusionError {
DataFusionError::Execution(format!("the node at {path} holds no binary operation"))
}
fn no_vector(path: &NodePath) -> DataFusionError {
DataFusionError::Execution(format!("the node at {path} holds no vector operation"))
}
/// The error for a node asked to broadcast without the broadcast its kind holds.
fn no_broadcast(path: &NodePath) -> DataFusionError {
DataFusionError::Execution(format!("the node at {path} holds no scalar broadcast"))
src/promql/src/native/execute/tests.rs
+461 -2
View File
@@ -40,14 +40,16 @@ use super::{
};
use crate::evaluated::EvaluatedChunk;
use crate::grid_series::merge_same_labelset;
use crate::histogram_eval::HistogramFunction;
use crate::instant_eval::select_instants;
use crate::map::MathOp;
use crate::native::metrics::{NodeMetric, NodeMetrics, NodePath};
use crate::native::plan::{
AggregateKind, AggregateStage, BoundAggregate, BoundInput, BoundInstant, BoundMap, BoundScalar,
BoundWindow, Grouping, NativeExpr, NativeNode, NativeTree, NativeType,
BoundVectorOp, BoundWindow, Grouping, NativeExpr, NativeNode, NativeTree, NativeType,
};
use crate::range_eval::{RangeEvaluator, RangeFunction, RangeWindow};
use crate::select::SortOrder;
use crate::vector_match::{
BinaryMatching, BinaryOp, Cardinality, FillValues, MatchingLabels, reset_visited_pairs,
visited_pairs,
@@ -480,11 +482,13 @@ fn source_runtime(pool: &Arc<dyn MemoryPool>) -> NodeRuntime {
pool,
),
queued: VecDeque::new(),
queued_tables: 0,
instant: None,
window: None,
map: None,
binary: None,
broadcast: None,
vector: None,
source_schema: None,
emitted: None,
merge: None,
@@ -497,6 +501,7 @@ fn binary_runtime(pool: &Arc<dyn MemoryPool>, schema: &Arc<SeriesSchema>) -> Nod
NodeRuntime {
buffer: NodeBuffer::new(&NodePath::root(), Arc::new(NodeMetrics::default()), pool),
queued: VecDeque::new(),
queued_tables: 0,
instant: None,
window: None,
map: None,
@@ -505,6 +510,7 @@ fn binary_runtime(pool: &Arc<dyn MemoryPool>, schema: &Arc<SeriesSchema>) -> Nod
[Arc::clone(schema), Arc::clone(schema)],
)),
broadcast: None,
vector: None,
source_schema: None,
emitted: None,
merge: None,
@@ -6153,7 +6159,8 @@ fn an_operand_chunk_that_rewinds_is_refused_with_or_without_values() {
let message = error.to_string();
assert!(
message.contains("do not continue each other")
&& message.contains("the left operand of the binary node at root")
&& message.contains("the binary node at root")
&& message.contains("by its left operand")
&& message.contains("after the series already reached 2"),
"{message}"
);
@@ -7243,3 +7250,455 @@ fn a_broadcast_refuses_two_series_that_share_a_label_set_and_releases_the_reserv
"error retained execution bytes: {message}"
);
}
fn vector_op(inner: NativeNode, op: BoundVectorOp) -> NativeNode {
let output = inner.output.clone();
NativeNode {
expr: NativeExpr::Vector {
input: Box::new(inner),
op,
},
output,
}
}
/// The positions a series decided, with their values.
type Decided = Vec<(usize, Option<f64>)>;
/// The values a chunk stream decided, by the `job` label of the series that holds them.
fn decided_by_job(chunks: &[EvaluatedChunk]) -> Vec<(String, Decided)> {
chunks
.iter()
.map(|chunk| {
(
chunk
.series
.metric
.get("job")
.unwrap_or_default()
.to_string(),
decided(std::slice::from_ref(chunk)),
)
})
.collect()
}
/// `topk` ranks the whole vector at each point, so a series that arrives in several chunks is one
/// candidate, and the reservation is given back once the kept series are handed over.
#[test]
fn topk_ranks_whole_series_at_each_point() {
let grid = grid(0, 60_000, 3);
let schema = evaluated_schema(Arc::clone(&grid));
let tree = NativeTree::try_new(vector_op(
evaluated_input(Arc::clone(&grid)),
BoundVectorOp::TopK {
k: 1.0,
top: true,
group: Grouping::By(Arc::from([])),
},
))
.unwrap();
let pool = pool();
let mut execution = executor(&tree, Arc::clone(&pool), &ExecutionPlanMetricsSet::new());
for chunk in [
evaluated_chunk_of(
&schema,
named_metric("m", "a"),
0,
vec![Some(1.0), Some(5.0)],
false,
),
evaluated_chunk_of(&schema, named_metric("m", "a"), 2, vec![Some(7.0)], true),
evaluated_chunk_of(
&schema,
named_metric("m", "b"),
0,
vec![Some(3.0), Some(4.0), Some(6.0)],
true,
),
] {
execution.push_evaluated(chunk).unwrap();
}
let chunks = run(&mut execution);
assert_eq!(
decided_by_job(&chunks),
vec![
("a".to_string(), vec![(1, Some(5.0)), (2, Some(7.0))]),
("b".to_string(), vec![(0, Some(3.0))]),
]
);
assert!(chunks.iter().all(|chunk| chunk.is_last));
assert_eq!(chunks[0].series.metric.name(), Some("m"));
assert_eq!(pool.reserved(), 0);
}
/// `limitk` takes series in the order its input hands them over, not in label order, and the two
/// series a map folds into `{job="z"}` are one series placed where it first arrived: it is kept at
/// both points although `{job="y"}` sorts first and arrived before the piece that fills point 0.
#[test]
fn limitk_takes_series_in_input_order() {
let grid = grid(0, 60_000, 2);
let schema = evaluated_schema(Arc::clone(&grid));
let tree = NativeTree::try_new(vector_op(
map(evaluated_input(Arc::clone(&grid)), vec![MathOp::Abs]),
BoundVectorOp::LimitK {
k: 1.0,
group: Grouping::By(Arc::from([])),
},
))
.unwrap();
let mut execution = executor(&tree, pool(), &ExecutionPlanMetricsSet::new());
for chunk in [
evaluated_chunk_of(&schema, named_metric("a", "z"), 1, vec![Some(1.0)], true),
evaluated_chunk_of(
&schema,
named_metric("b", "y"),
0,
vec![Some(2.0), Some(2.0)],
true,
),
evaluated_chunk_of(&schema, named_metric("c", "z"), 0, vec![Some(3.0)], true),
] {
execution.push_evaluated(chunk).unwrap();
}
let chunks = run(&mut execution);
assert_eq!(
decided_by_job(&chunks),
vec![("z".to_string(), vec![(0, Some(3.0)), (1, Some(1.0))])]
);
}
/// A binary node above a vector node reads what `topk` kept, once the vector node has read its
/// whole input: `topk` keeps `{job="b"}` at point 0 and `{job="a"}` at point 1, so only point 1
/// matches the right side's `{job="a"}`.
#[test]
fn a_binary_node_reads_what_a_vector_node_kept() {
let grid = grid(0, 60_000, 2);
let schema = evaluated_schema(Arc::clone(&grid));
let tree = NativeTree::try_new(binary_of(
vector_op(
evaluated_input(Arc::clone(&grid)),
BoundVectorOp::TopK {
k: 1.0,
top: true,
group: Grouping::By(Arc::from([])),
},
),
evaluated_input(Arc::clone(&grid)),
binary_matching(BinaryOp::Add),
))
.unwrap();
let (left, right) = (NodePath::root().left().input(), NodePath::root().right());
let mut execution = executor(&tree, pool(), &ExecutionPlanMetricsSet::new());
for (path, chunk) in [
(
&left,
evaluated_chunk_of(
&schema,
named_metric("m", "a"),
0,
vec![Some(1.0), Some(5.0)],
true,
),
),
(
&right,
evaluated_chunk_of(
&schema,
named_metric("m", "a"),
0,
vec![Some(10.0), Some(10.0)],
true,
),
),
(
&left,
evaluated_chunk_of(
&schema,
named_metric("m", "b"),
0,
vec![Some(3.0), Some(4.0)],
true,
),
),
] {
execution.push_evaluated_at(path, chunk).unwrap();
}
assert_eq!(decided(&run(&mut execution)), vec![(1, Some(15.0))]);
}
/// `absent` answers where its input holds nothing, under the labels it was bound to, and answers
/// at every point when no series arrives at all.
#[test]
fn absent_answers_where_the_input_holds_nothing() {
let grid = grid(0, 60_000, 3);
let schema = evaluated_schema(Arc::clone(&grid));
// What `absent_labels` derives from `up{job="x"}`: the name is not one of them.
let labels = MetricName::try_new(
None,
vec![Label {
name: "job".into(),
value: "x".into(),
}],
)
.unwrap();
let tree = NativeTree::try_new(vector_op(
evaluated_input(Arc::clone(&grid)),
BoundVectorOp::Absent {
labels: labels.clone(),
},
))
.unwrap();
let mut execution = executor(&tree, pool(), &ExecutionPlanMetricsSet::new());
execution
.push_evaluated(evaluated_chunk_of(
&schema,
named_metric("up", "y"),
1,
vec![Some(0.0)],
true,
))
.unwrap();
let chunks = run(&mut execution);
assert_eq!(chunks.len(), 1);
assert_eq!(chunks[0].series.metric, labels);
assert_eq!(decided(&chunks), vec![(0, Some(1.0)), (2, Some(1.0))]);
let mut execution = executor(&tree, pool(), &ExecutionPlanMetricsSet::new());
assert_eq!(
decided(&run(&mut execution)),
vec![(0, Some(1.0)), (1, Some(1.0)), (2, Some(1.0))]
);
}
/// A sort orders the series of an instant result, and the node hands them over in that order.
#[test]
fn sort_desc_hands_over_series_in_value_order() {
let grid = grid(0, 60_000, 1);
let schema = evaluated_schema(Arc::clone(&grid));
let tree = NativeTree::try_new(vector_op(
evaluated_input(Arc::clone(&grid)),
BoundVectorOp::Sort(SortOrder::Desc),
))
.unwrap();
let mut execution = executor(&tree, pool(), &ExecutionPlanMetricsSet::new());
for (job, value) in [("b", 1.0), ("a", 3.0), ("c", 2.0)] {
execution
.push_evaluated(evaluated_chunk_of(
&schema,
named_metric("m", job),
0,
vec![Some(value)],
true,
))
.unwrap();
}
let jobs: Vec<String> = run(&mut execution)
.iter()
.map(|chunk| chunk.series.metric.get("job").unwrap().to_string())
.collect();
assert_eq!(jobs, ["a", "c", "b"]);
}
/// A sort hands back the assembled input, whose field is one slice per input chunk of one shared
/// buffer: it is queued at what the slices hold, as the input was retained, not at the buffer once
/// per chunk, which here would be ten times the input and more than the pool.
#[test]
fn a_sorted_series_is_queued_at_what_its_slices_hold() {
let points = 1_000;
let grid = grid(0, 60_000, points);
let schema = evaluated_schema(Arc::clone(&grid));
let tree = NativeTree::try_new(vector_op(
evaluated_input(Arc::clone(&grid)),
BoundVectorOp::Sort(SortOrder::ByLabel(vec!["job".into()])),
))
.unwrap();
let pool = bounded_pool(48 * 1024);
let mut execution = executor(&tree, Arc::clone(&pool), &ExecutionPlanMetricsSet::new());
let values = Float64Array::from((0..points).map(|v| v as f64).collect::<Vec<_>>());
for start in (0..points as usize).step_by(100) {
execution
.push_evaluated(evaluated_array_chunk_of(
&schema,
named_metric("m", "a"),
start,
values.slice(start, 100),
start + 100 == points as usize,
))
.unwrap();
// The vector node retains each chunk as it arrives, at what its slice holds.
assert!(execution.next_chunk().unwrap().is_none());
}
let chunks = run(&mut execution);
assert_eq!(decided(&chunks).len(), points as usize);
assert_eq!(pool.reserved(), 0);
}
/// Two sorted results share one histogram table, so the table stays charged until the second one
/// leaves, not only until the first one does.
#[test]
fn a_table_shared_by_queued_results_stays_charged_until_the_last_leaves() {
let grid = grid(0, 60_000, 1);
let schema = evaluated_schema(Arc::clone(&grid));
let tree = NativeTree::try_new(vector_op(
evaluated_input(Arc::clone(&grid)),
BoundVectorOp::Sort(SortOrder::ByLabel(vec!["job".into()])),
))
.unwrap();
let table = histogram_table(2, 64);
let pool = pool();
let mut execution = executor(&tree, Arc::clone(&pool), &ExecutionPlanMetricsSet::new());
for (job, index) in [("a", 0), ("b", 1)] {
execution
.push_evaluated(evaluated_histogram_chunk_of(
&schema,
named_metric("m", job),
0,
vec![Some(index)],
&table,
true,
))
.unwrap();
}
execution.end_of_input().unwrap();
let first = execution.next_chunk().unwrap().expect("two sorted results");
assert_eq!(first.series.metric.get("job"), Some("a"));
drop(first);
assert!(pool.reserved() >= histogram_table_bytes(&table));
let second = execution.next_chunk().unwrap().expect("two sorted results");
assert_eq!(second.series.metric.get("job"), Some("b"));
assert!(execution.next_chunk().unwrap().is_none());
assert_eq!(pool.reserved(), 0);
}
/// `limitk` takes the order a sort below it decided: `sort_desc` puts `{job="z"}` first, so it is
/// the one `limitk(1, ...)` keeps, although `{job="a"}` sorts first by label.
#[test]
fn limitk_keeps_the_order_a_sort_below_it_decided() {
let grid = grid(0, 60_000, 1);
let schema = evaluated_schema(Arc::clone(&grid));
let tree = NativeTree::try_new(vector_op(
vector_op(
evaluated_input(Arc::clone(&grid)),
BoundVectorOp::Sort(SortOrder::Desc),
),
BoundVectorOp::LimitK {
k: 1.0,
group: Grouping::By(Arc::from([])),
},
))
.unwrap();
let mut execution = executor(&tree, pool(), &ExecutionPlanMetricsSet::new());
for (job, value) in [("a", 1.0), ("z", 2.0)] {
execution
.push_evaluated(evaluated_chunk_of(
&schema,
named_metric("v", job),
0,
vec![Some(value)],
true,
))
.unwrap();
}
assert_eq!(
decided_by_job(&run(&mut execution)),
vec![("z".to_string(), vec![(0, Some(2.0))])]
);
}
/// `histogram_quantile` reads a classic histogram from the bucket series that share its labels,
/// however they arrived, and answers under those labels without `le` and the name.
#[test]
fn histogram_quantile_reads_the_buckets_of_the_whole_vector() {
let grid = grid(0, 60_000, 1);
let schema = evaluated_schema(Arc::clone(&grid));
let tree = NativeTree::try_new(vector_op(
evaluated_input(Arc::clone(&grid)),
BoundVectorOp::Histogram(HistogramFunction::Quantile(0.5)),
))
.unwrap();
let bucket = |le: &str| {
MetricName::try_new(
Some("m".into()),
vec![
Label {
name: "job".into(),
value: "a".into(),
},
Label {
name: "le".into(),
value: le.into(),
},
],
)
.unwrap()
};
let mut execution = executor(&tree, pool(), &ExecutionPlanMetricsSet::new());
for (le, count) in [("+Inf", 2.0), ("1", 1.0)] {
execution
.push_evaluated(evaluated_chunk_of(
&schema,
bucket(le),
0,
vec![Some(count)],
true,
))
.unwrap();
}
let chunks = run(&mut execution);
assert_eq!(chunks.len(), 1);
assert_eq!(chunks[0].series.metric.name(), None);
assert_eq!(chunks[0].series.metric.get("le"), None);
assert_eq!(chunks[0].series.metric.get("job"), Some("a"));
assert_eq!(decided(&chunks), vec![(0, Some(1.0))]);
}
/// An empty tag and an absent one are one label set, so the two native histogram series below are
/// one series to `histogram_quantile`, answered at both of their points rather than one of them
/// dropped.
#[test]
fn histogram_quantile_reads_the_pieces_of_one_label_set_as_one_series() {
let grid = grid(0, 60_000, 2);
let schema = evaluated_schema(Arc::clone(&grid));
let tree = NativeTree::try_new(vector_op(
evaluated_input(Arc::clone(&grid)),
BoundVectorOp::Histogram(HistogramFunction::Quantile(0.5)),
))
.unwrap();
let metric = |tags: &[(&str, &str)]| {
MetricName::try_new(
Some("m".into()),
tags.iter()
.map(|(name, value)| Label {
name: (*name).into(),
value: (*value).into(),
})
.collect(),
)
.unwrap()
};
let table = histogram_table(2, 4);
let mut execution = executor(&tree, pool(), &ExecutionPlanMetricsSet::new());
for (start, tags, index) in [
(0, vec![("job", "a"), ("zone", "")], 0),
(1, vec![("job", "a")], 1),
] {
execution
.push_evaluated(evaluated_histogram_chunk_of(
&schema,
metric(&tags),
start,
vec![Some(index)],
&table,
true,
))
.unwrap();
}
let chunks = run(&mut execution);
let positions: Vec<usize> = decided(&chunks)
.into_iter()
.filter_map(|(position, value)| value.map(|_| position))
.collect();
assert_eq!(positions, [0, 1]);
}
src/promql/src/native/metrics.rs
+2 -1
View File
@@ -100,7 +100,8 @@ pub fn walk(root: &NativeNode, visit: &mut dyn FnMut(&NativeNode, &NodePath)) {
NativeExpr::Instant { input, .. }
| NativeExpr::Window { input, .. }
| NativeExpr::Map { input, .. }
| NativeExpr::Aggregate { input, .. } => go(input, &path.input(), visit),
| NativeExpr::Aggregate { input, .. }
| NativeExpr::Vector { input, .. } => go(input, &path.input(), visit),
NativeExpr::Binary { left, right, .. } => {
go(left, &path.left(), visit);
go(right, &path.right(), visit);
src/promql/src/native/plan.rs
+49 -1
View File
@@ -31,12 +31,14 @@
use std::sync::Arc;
use datafusion::error::{DataFusionError, Result};
use datatypes::series::SeriesOrdering;
use datatypes::series::samples::{FieldSpec, SeriesSchema};
use datatypes::series::time::EvalGrid;
use datatypes::series::{MetricName, SeriesOrdering};
use crate::histogram_eval::HistogramFunction;
use crate::map::MathOp;
use crate::range_eval::{RangeFunction, RangeWindow};
use crate::select::SortOrder;
use crate::vector_match::BinaryMatching;
/// What one node produces.
@@ -255,6 +257,33 @@ pub struct BoundAggregate {
pub stage: AggregateStage,
}
/// An operator over the whole input vector at each grid point: one that keeps some of the input's
/// samples with their series' labels and values, orders the series, reports where none is present,
/// or reads a classic histogram from its bucket series.
///
/// Its constants are bound here, as a window's are: a `k` or a ratio only known at run time has no
/// field.
#[derive(Clone, Debug, PartialEq)]
pub enum BoundVectorOp {
/// `topk` (`top`) or `bottomk`: the `k` largest or smallest values of each group.
TopK { k: f64, top: bool, group: Grouping },
/// `limitk`: the first `k` series of each group that hold a sample, in input order.
LimitK { k: f64, group: Grouping },
/// `limit_ratio`: the series whose label hash falls in the ratio; it keeps whole series, so it
/// has no grouping.
LimitRatio { ratio: f64 },
/// `sort`, `sort_desc`, `sort_by_label` and `sort_by_label_desc`: the input's series in another
/// order, which only an instant result shows. `sort` and `sort_desc` also drop histogram
/// samples, in a range result too.
Sort(SortOrder),
/// `absent`: `1` under `labels` at every point where the input holds no sample. The labels are
/// what the compiler derives from the argument's matchers.
Absent { labels: MetricName },
/// `histogram_quantile` or `histogram_fraction`: each classic histogram is read from the bucket
/// series that share its labels, so this too reads the whole vector at a point.
Histogram(HistogramFunction),
}
/// What a node computes. Every variant holds a bound operation, never a name to resolve.
#[derive(Clone, Debug)]
pub enum NativeExpr {
@@ -282,6 +311,11 @@ pub enum NativeExpr {
input: Box<NativeNode>,
op: BoundAggregate,
},
/// An operation over the whole input vector at each grid point.
Vector {
input: Box<NativeNode>,
op: BoundVectorOp,
},
/// A binary operation between two inputs, with its matching rules.
///
/// Scalar operands are broadcast, so `matching` only applies to vector–vector operations.
@@ -423,6 +457,20 @@ impl NativeNode {
ensure_same_fields(&input.output, &self.output, "a map")?;
ensure_grid(&self.output, input_grid, "a map")?;
}
NativeExpr::Vector { input, .. } => {
input.validate()?;
// A vector operation keeps the input's fields and grid, so it consumes and produces
// the same evaluated vector.
let Some((_, input_grid)) = evaluated_vector(&input.output) else {
return Err(invalid(&format!(
"a vector operation consumes an evaluated vector, not {}",
input.output.describe()
)));
};
ensure_samples(&self.output, "a vector operation")?;
ensure_same_fields(&input.output, &self.output, "a vector operation")?;
ensure_grid(&self.output, input_grid, "a vector operation")?;
}
NativeExpr::Aggregate { input, op } => {
input.validate()?;
// A stage's contract is a pair of types, not only what it produces: `Partial`
src/promql/src/native/plan/tests.rs
+16 -1
View File
@@ -20,7 +20,7 @@ use datatypes::series::time::{EvalGrid, GridSpec};
use super::{
AggregateKind, AggregateStage, BoundAggregate, BoundInput, BoundInstant, BoundMap, BoundScalar,
BoundWindow, Grouping, NativeExpr, NativeNode, NativeTree, NativeType,
BoundVectorOp, BoundWindow, Grouping, NativeExpr, NativeNode, NativeTree, NativeType,
};
use crate::map::MathOp;
use crate::range_eval::{RangeFunction, RangeWindow};
@@ -1221,3 +1221,18 @@ fn an_input_can_bind_aggregate_state() {
let error = other.validate().unwrap_err();
assert!(error.to_string().contains("not the binding"), "{error}");
}
/// A vector operation reads the values at each grid point, so raw samples, which have no shared points,
/// are not something it can consume.
#[test]
fn a_vector_operation_cannot_consume_raw_samples() {
let node = NativeNode {
expr: NativeExpr::Vector {
input: Box::new(input()),
op: BoundVectorOp::LimitRatio { ratio: 0.5 },
},
output: evaluated(Arc::new([]), grid(0, 60_000, 4)),
};
let error = node.validate().unwrap_err();
assert!(error.to_string().contains("an evaluated vector"), "{error}");
}
src/promql/src/select.rs
+41 -5
View File
@@ -59,7 +59,8 @@ pub enum SortOrder {
/// `topk(k, ...)` (`top`) or `bottomk(k, ...)` with a constant `k`.
///
/// At every point each group keeps the `k` largest (or smallest) values, NaN ranking last for
/// both; the selected samples keep their series' labels and values. Prometheus breaks ties by
/// both; the selected samples keep their series' labels and values. Histogram samples are not
/// ranked, as Prometheus ignores them here. Prometheus breaks ties by
/// the order its storage returns series in, which is not a property of the series; here ties
/// go to the series whose label set sorts first.
pub fn topk(
@@ -102,7 +103,10 @@ pub fn topk(
let Some(row) = series[member].time.row_of(point) else {
continue;
};
if let Some(value) = cursor_value(&mut cursors[slot], row)? {
if let SampleRef::Float(value) = cursors[slot]
.at(row)
.expect("the axis and the fields are aligned")
{
candidates.push((member, row, value));
}
}
@@ -420,20 +424,30 @@ pub fn absent_labels(matchers: &[LabelMatcher]) -> Result<MetricName> {
MetricName::try_new(None, labels).map_err(external)
}
/// Orders the series of an instant result. A range result is returned unchanged: Prometheus
/// does not sort matrices.
/// Orders the series of an instant result. A range result keeps its order: Prometheus does not
/// sort matrices.
///
/// `sort` and `sort_desc` drop histogram samples, in a range result too: Prometheus filters them
/// out at every step (`filterFloats`) before it sorts. The label sorts keep every sample.
pub fn sort_instant(
order: &SortOrder,
schema: &SeriesSchema,
mut series: Vec<Series>,
) -> Result<Vec<Series>> {
let points = grid_points(schema, &series)?;
if matches!(order, SortOrder::Asc | SortOrder::Desc) {
single_field(schema, "sort")?;
let mut floats = Vec::with_capacity(series.len());
for item in series {
floats.extend(float_samples(schema, item)?);
}
series = floats;
}
if points != 1 {
return Ok(series);
}
match order {
SortOrder::Asc | SortOrder::Desc => {
single_field(schema, "sort")?;
let mut keyed = Vec::with_capacity(series.len());
for item in series {
let value = value_at(&item, 0)?;
@@ -472,6 +486,28 @@ pub fn sort_instant(
}
}
/// `series` without its histogram samples, `None` when nothing else is left. One field, as `sort`
/// reads.
fn float_samples(schema: &SeriesSchema, series: Series) -> Result<Option<Series>> {
let field = &series.fields.as_slice()[0];
if !field
.iter()
.any(|sample| matches!(sample, SampleRef::Histogram(_)))
{
return Ok(Some(series));
}
let mut builder = GridSeriesBuilder::try_new(Arc::new(schema.clone()))?;
let mut cursor = field.cursor();
for (point, row) in series.time.positions() {
let value = match cursor.at(row).expect("the axis and the fields are aligned") {
SampleRef::Float(value) => Some(value),
SampleRef::Missing | SampleRef::Histogram(_) => None,
};
builder.push(point, &[value])?;
}
builder.finish(series.metric)
}
/// Go's `math.NaN()` bit pattern, which Prometheus returns for an undefined scalar.
const GO_NAN: f64 = f64::from_bits(0x7FF8_0000_0000_0001);
src/promql/src/select/tests.rs
+70
View File
@@ -269,6 +269,76 @@ fn limit_operators_keep_histogram_samples() {
assert_eq!(kinds(&kept[0], 0), [(1, "histogram")]);
}
/// `topk` ranks floats only: a histogram sample is skipped, as Prometheus ignores it, and the
/// float below it is kept in its place.
#[test]
fn topk_skips_histogram_samples() {
let histogram =
OwnedHistogram::parse("0/0/0/3ff0000000000000/3ff0000000000000/0/1_1/3ff0000000000000///");
let schema = schema(1, 1);
let mut builder = GridSeriesBuilder::try_new(Arc::new(schema.clone())).unwrap();
builder
.push_samples(
0,
&[Some(GridSample::Histogram(Box::new(
FloatHistogram::try_from_view(&histogram.view()).unwrap(),
)))],
)
.unwrap();
let series = vec![
builder.finish(metric("__name__=a")).unwrap().unwrap(),
build(&schema, metric("__name__=b"), &vec![vec![Some(1.0)]]).unwrap(),
];
let kept = topk(1.0, true, &Grouping::By(Vec::new()), &schema, &series).unwrap();
assert_eq!(kept.len(), 1);
assert_eq!(kept[0].metric.name(), Some("b"));
}
/// `sort` drops histogram samples at every step, as Prometheus' `filterFloats` does, while
/// `sort_by_label` keeps them.
#[test]
fn sort_drops_histogram_samples() {
let histogram =
OwnedHistogram::parse("0/0/0/3ff0000000000000/3ff0000000000000/0/1_1/3ff0000000000000///");
let mixed = |schema: &SeriesSchema, name: &str| {
let mut builder = GridSeriesBuilder::try_new(Arc::new(schema.clone())).unwrap();
builder
.push_samples(
0,
&[Some(GridSample::Histogram(Box::new(
FloatHistogram::try_from_view(&histogram.view()).unwrap(),
)))],
)
.unwrap();
if schema.eval_grid.as_ref().unwrap().spec().len > 1 {
builder
.push_samples(1, &[Some(GridSample::Float(2.0))])
.unwrap();
}
builder
.finish(metric(&format!("__name__={name}")))
.unwrap()
.unwrap()
};
let instant = schema(1, 1);
let series = vec![
mixed(&instant, "h"),
build(&instant, metric("__name__=f"), &vec![vec![Some(1.0)]]).unwrap(),
];
let sorted = sort_instant(&SortOrder::Asc, &instant, series.clone()).unwrap();
assert_eq!(sorted.len(), 1);
assert_eq!(sorted[0].metric.name(), Some("f"));
let by_name = sort_instant(&SortOrder::ByLabel(names("__name__")), &instant, series).unwrap();
assert_eq!(by_name.len(), 2);
let range = schema(2, 1);
let sorted = sort_instant(&SortOrder::Desc, &range, vec![mixed(&range, "h")]).unwrap();
assert_eq!(sorted.len(), 1);
let positions: Vec<usize> = sorted[0].time.positions().map(|(point, _)| point).collect();
assert_eq!(positions, [1]);
}
/// A selection hands back the sample it selected, start timestamp included.
///
/// `limitk` and `limit_ratio` choose samples rather than compute them, so a histogram that arrives
src/query/src/promql/native_compile.rs
+178 -5
View File
@@ -27,6 +27,9 @@
//! double_exponential_smoothing(<matrix>, <constant>, <constant>)
//! e.g. double_exponential_smoothing(x[5m], .5, .1)
//! <math function>(<expression>[, <constant>]*) e.g. abs(rate(x[5m])), clamp_min(x, 0)
//! <sort>(<expression>[, <string>]*) e.g. sort_desc(x), sort_by_label(x, "job")
//! absent(<expression>) e.g. absent(x{job="a"})
//! <histogram>(<constant>+, <expression>) e.g. histogram_quantile(0.9, rate(x[5m]))
//! <expression> <operator> <constant> e.g. x * 100, x > bool 0
//! <constant> <operator> <expression> e.g. 1 / x, 0 < x
//! -<expression> e.g. -x
@@ -119,22 +122,26 @@ use std::sync::Arc;
use std::time::Duration;
use datafusion::error::{DataFusionError, Result};
use datatypes::series::SeriesOrdering;
use datatypes::series::samples::SeriesSchema;
use datatypes::series::time::{EvalGrid, GridSpec};
use datatypes::series::{MetricName, SeriesOrdering};
use promql::histogram_eval::HistogramFunction;
use promql::map::{ArithmeticOp, ComparisonOp, MathOp};
use promql::map_time::DateFunction;
use promql::native::plan::{
AggregateKind, AggregateStage, BoundAggregate, BoundInput, BoundInstant, BoundMap, BoundScalar,
BoundWindow, Grouping, NativeExpr, NativeNode, NativeTree, NativeType,
BoundVectorOp, BoundWindow, Grouping, NativeExpr, NativeNode, NativeTree, NativeType,
};
use promql::range_eval::{RangeFunction, RangeWindow, subquery_grid};
use promql::select::{LabelMatcher, MatchOp, SortOrder, absent_labels};
use promql::vector_match::{BinaryMatching, BinaryOp, Cardinality, FillValues, MatchingLabels};
use promql_parser::label::MatchOp as ParserMatchOp;
use promql_parser::parser::token::TokenType;
use promql_parser::parser::value::ValueType;
use promql_parser::parser::{
AggregateExpr, AtModifier, BinModifier, BinaryExpr, Call, EvalStmt, Expr as PromExpr,
LabelModifier, MatrixSelector, NumberLiteral, Offset, SubqueryExpr, VectorMatchCardinality,
VectorMatchFillValues, VectorSelector, token,
LabelModifier, MatrixSelector, NumberLiteral, Offset, StringLiteral, SubqueryExpr,
VectorMatchCardinality, VectorMatchFillValues, VectorSelector, token,
};
use crate::promql::label_values::signed_millis_since_epoch;
@@ -159,7 +166,10 @@ const SUPPORTED: &str = "a selector — `x`, optionally with an `offset` — a r
`count`, `min`, `max`, `group`, `stddev`, `stdvar` and \
`quantile(<constant>, <expression>)` — over any of them, optionally \
grouped by `by` or `without` as in `avg by(job)(rate(x[5m]) * 100)`; a \
range function's own constants sit where that function takes them, before \
sort, `absent` or `histogram_quantile` / `histogram_fraction` over any of \
them (`sort_desc(x)`, `absent(x{job=\"a\"})`, \
`histogram_quantile(0.9, rate(x[5m]))`); a range function's own \
constants sit where that function takes them, before \
the matrix for `quantile_over_time(0.9, x[5m])` and after it for \
`predict_linear(x[5m], 300)` and \
`double_exponential_smoothing(x[5m], 0.5, 0.1)`";
@@ -580,6 +590,9 @@ fn compile_call(context: &Context, call: &Call) -> Result<NativeNode> {
if let Some(signature) = range_signature(func.name) {
return window_call(context, func.name, signature, &args.args);
}
if let Some((op, vector)) = vector_op(func.name, &args.args)? {
return vector_call(context, func.name, op, vector);
}
// The name is looked up before the call's arguments are read, so a name this compiler has no
// kernel for is refused as such whichever argument it was given: `absent(x)` is refused as a
// name with no kernel, not as a selector this compiler could not compile.
@@ -672,6 +685,148 @@ fn map_call(context: &Context, name: &str, op: MathOp, vector: &PromExpr) -> Res
})
}
/// Wraps the vector operation `op` over the value of `vector`, which keeps its type as a map does.
fn vector_call(
context: &Context,
name: &str,
op: BoundVectorOp,
vector: &PromExpr,
) -> Result<NativeNode> {
if vector.value_type() != ValueType::Vector {
return Err(not_implemented(&format!(
"a call to `{name}` whose vector argument is `{vector}`, which is a {}, where the \
function takes a vector",
vector.value_type()
)));
}
let input = compile_expr(context, vector)?;
let output = input.output.clone();
Ok(NativeNode {
expr: NativeExpr::Vector {
input: Box::new(input),
op,
},
output,
})
}
/// The vector operation a sort, `absent` or a classic histogram function binds, and the vector it reads,
/// or `None` when `name` is none of them.
fn vector_op<'a>(
name: &str,
arguments: &'a [Box<PromExpr>],
) -> Result<Option<(BoundVectorOp, &'a PromExpr)>> {
let order = match name {
"sort" => Some(SortOrder::Asc),
"sort_desc" => Some(SortOrder::Desc),
"sort_by_label" | "sort_by_label_desc" => None,
"histogram_quantile" | "histogram_fraction" => {
let Some((vector, constants)) = arguments.split_last() else {
return Err(not_implemented(&format!(
"a call to `{name}` with no argument"
)));
};
let function = match constants {
[quantile] if name == "histogram_quantile" => {
HistogramFunction::Quantile(constant_argument(name, quantile, 0)?)
}
[lower, upper] if name == "histogram_fraction" => HistogramFunction::Fraction {
lower: constant_argument(name, lower, 0)?,
upper: constant_argument(name, upper, 1)?,
},
_ => {
return Err(not_implemented(&format!(
"a call to `{name}` with {} argument(s)",
arguments.len()
)));
}
};
return Ok(Some((BoundVectorOp::Histogram(function), vector)));
}
"absent" => {
let [vector] = arguments else {
return Err(not_implemented(&format!(
"a call to `absent` with {} argument(s)",
arguments.len()
)));
};
let labels = absent_labels_of(vector)?;
return Ok(Some((BoundVectorOp::Absent { labels }, vector)));
}
_ => return Ok(None),
};
let Some((vector, strings)) = arguments.split_first() else {
return Err(not_implemented(&format!(
"a call to `{name}` with no argument"
)));
};
let order = match order {
Some(order) if strings.is_empty() => order,
Some(_) => {
return Err(not_implemented(&format!(
"a call to `{name}` with {} argument(s)",
arguments.len()
)));
}
None => {
let labels = strings
.iter()
.map(|argument| match unparen(argument) {
PromExpr::StringLiteral(StringLiteral { val }) => Ok(Arc::from(val.as_str())),
other => Err(not_implemented(&format!(
"a call to `{name}` whose argument `{other}` is not a string literal"
))),
})
.collect::<Result<Vec<_>>>()?;
if name == "sort_by_label" {
SortOrder::ByLabel(labels)
} else {
SortOrder::ByLabelDesc(labels)
}
}
};
Ok(Some((BoundVectorOp::Sort(order), vector)))
}
/// The labels `absent(expr)` answers with, as Prometheus `createLabelsForAbsentFunction` derives
/// them: from the matchers of a vector selector, and none for any other expression.
///
/// `__field__`, `__schema__` and `__database__` choose what is read rather than match a label, so
/// they are not labels of the answer, as in the existing planner.
fn absent_labels_of(expr: &PromExpr) -> Result<MetricName> {
let PromExpr::VectorSelector(selector) = unparen(expr) else {
return absent_labels(&[]);
};
if !selector.matchers.or_matchers.is_empty() {
return Err(not_implemented(&format!(
"`absent` over `{selector}`, whose matchers are alternatives"
)));
}
let matchers = selector
.matchers
.matchers
.iter()
.filter(|matcher| {
!matches!(
matcher.name.as_str(),
"__field__" | "__schema__" | "__database__"
)
})
.map(|matcher| LabelMatcher {
name: Arc::from(matcher.name.as_str()),
op: match matcher.op {
ParserMatchOp::Equal => MatchOp::Equal,
ParserMatchOp::NotEqual => MatchOp::NotEqual,
ParserMatchOp::Re(_) => MatchOp::Regex,
ParserMatchOp::NotRe(_) => MatchOp::NotRegex,
},
value: Arc::from(matcher.value.as_str()),
matches_empty: matcher.is_match(""),
})
.collect::<Vec<_>>();
absent_labels(&matchers)
}
/// Binds `function` over the range selector `vs[range]` into a window over the raw input.
///
/// Both shapes reach this: the range, the offset, the anchor, the grid and the output are the same
@@ -1158,6 +1313,18 @@ fn math_signature(name: &str) -> Option<MathSignature> {
"deg" => Some(MathSignature::Unary(MathOp::Deg)),
"rad" => Some(MathSignature::Unary(MathOp::Rad)),
"sgn" => Some(MathSignature::Unary(MathOp::Sgn)),
// A date function takes the vector its values are read from; without one it reads the
// evaluation time, which has no input to map and is refused by `math_op`.
"day_of_month" => Some(MathSignature::Unary(MathOp::Date(DateFunction::DayOfMonth))),
"day_of_week" => Some(MathSignature::Unary(MathOp::Date(DateFunction::DayOfWeek))),
"day_of_year" => Some(MathSignature::Unary(MathOp::Date(DateFunction::DayOfYear))),
"days_in_month" => Some(MathSignature::Unary(MathOp::Date(
DateFunction::DaysInMonth,
))),
"hour" => Some(MathSignature::Unary(MathOp::Date(DateFunction::Hour))),
"minute" => Some(MathSignature::Unary(MathOp::Date(DateFunction::Minute))),
"month" => Some(MathSignature::Unary(MathOp::Date(DateFunction::Month))),
"year" => Some(MathSignature::Unary(MathOp::Date(DateFunction::Year))),
// `round`'s nearest defaults to one, so `round(x)` is `round(x, 1)` and its variant always
// carries the number it rounds to.
"round" => Some(MathSignature::DefaultedConstant {
@@ -1195,6 +1362,12 @@ fn math_op<'a>(
// The vector is the function's first argument, so a call that wrote no argument at all is
// refused by the same arity check as one that wrote too few constants.
let Some((vector, parameters)) = arguments.split_first() else {
if let MathSignature::Unary(MathOp::Date(_)) = signature {
return Err(not_implemented(&format!(
"a call to `{name}` without an argument, which reads the evaluation time and has \
no native kernel"
)));
}
return Err(arity_error(name, signature, 0));
};
let given = parameters.len() + 1;
src/query/src/promql/native_compile/tests.rs
+57 -3
View File
@@ -537,6 +537,9 @@ fn a_call_with_the_wrong_number_of_arguments_names_the_count_it_takes() {
"abs",
"with 0 argument(s), where it takes its vector",
),
// A date function written without an argument is valid PromQL that reads the evaluation
// time; there is no input to map, so it is left to the ordinary path.
(eval_stmt("hour()"), "hour", "reads the evaluation time"),
];
for (stmt, name, expected) in cases {
@@ -937,9 +940,9 @@ fn an_unsupported_shape_is_rejected_and_names_what_it_found() {
// A function with no kernel. `predict_linear` used to stand here as the parameterised case
// and is now bound, so what is left is a name this compiler has no kernel for at all.
(
eval_stmt("absent(x)"),
"absent(x)",
"a call to `absent`, which has no native kernel",
eval_stmt("timestamp(x)"),
"timestamp(x)",
"a call to `timestamp`, which has no native kernel",
),
];
@@ -1212,6 +1215,15 @@ fn the_supported_math_functions_map_to_their_kernels() {
("clamp(x, 0, 1)", "clamp 0 1"),
("clamp_min(x, 0)", "clamp_min 0"),
("clamp_max(x, 1)", "clamp_max 1"),
// Date functions over a vector read its values as Unix seconds.
("day_of_month(x)", "day_of_month"),
("day_of_week(x)", "day_of_week"),
("day_of_year(x)", "day_of_year"),
("days_in_month(x)", "days_in_month"),
("hour(x)", "hour"),
("minute(x)", "minute"),
("month(x)", "month"),
("year(x)", "year"),
];
for (query, op) in cases {
@@ -3669,3 +3681,45 @@ fn a_subquery_grid_holds_every_sample_its_windows_read() {
);
}
}
/// The sorts, `absent` and the classic histogram functions bind a vector operation over the vector they
/// are given, the histogram functions with their constants folded. `absent` derives its
/// labels from the selector's matchers as Prometheus does: the first equality of a name sets the
/// label and any other matcher of it removes it, the matchers that accept a missing label read
/// last; `__name__` and the table-choosing matchers are not labels, and any expression other than
/// a selector answers without labels.
#[test]
fn whole_vector_functions_bind_a_vector_operation() {
let cases = [
("sort(x)", "sort"),
("sort_desc(rate(x[5m]))", "sort_desc"),
(
r#"sort_by_label(x, "job", "le")"#,
"sort_by_label [job, le]",
),
(
r#"sort_by_label_desc(x, "job")"#,
"sort_by_label_desc [job]",
),
(
r#"absent(x{job="a", env=~"p.*", instance!="", __field__="v"})"#,
r#"absent {job="a"}"#,
),
// `job!="b"` accepts a missing `job`, so it reads after `job="a"` and removes it.
(r#"absent(x{job!="b", job="a"})"#, "absent {}"),
(r#"absent({__name__="x", job="a"})"#, r#"absent {job="a"}"#),
(
"histogram_quantile(1 - 0.1, rate(x[5m]))",
"histogram_quantile 0.9",
),
("histogram_fraction(0, 0.5, x)", "histogram_fraction 0 0.5"),
("absent(rate(x[5m]))", "absent {}"),
];
for (query, op) in cases {
let tree = compile(&eval_stmt(query), raw_schema(), SeriesOrdering::KeyThenTime)
.unwrap_or_else(|error| panic!("`{query}` was refused: {error}"));
let display = tree.to_string();
let root = display.lines().nth(1).expect("a compiled tree has a root");
assert_eq!(root, format!(" root Vector {op}"), "`{query}`");
}
}
tests/compatibility/wire/mito_columnar_samples/README.md
+39
View File
@@ -0,0 +1,39 @@
# Mito columnar samples version 1
This fixture freezes the version-1 sample wire contract in
[`mito-samples.md`](../../../../docs/formats/mito-samples.md). It is exercised by
the owning Mito Rust test, not SQLness discovery:
```shell
cargo nextest run -p mito2 frozen_v1_samples_fixture_and_unknown_codec
```
`v1-manifest.json` and `v1-part.hex` describe a 397-byte part with raw compression,
one leaf/root, and two samples for `(table_id=7, tsid=42)` at timestamp 100 ms.
The first row is a put with sequence 9 and Float64 negative zero. The second is a
delete with sequence 8 and a null field. Both rows must survive decoding.
The part was assembled independently of the Rust writer, using explicit
little-endian fields and XXH64 with seed zero. Its layout is:
| Range | Content |
| --- | --- |
| 0..16 | Header |
| 16..32 | Constant-step timestamps: first 100, step 0 |
| 32..41 | Delta sequences: first 9, ZigZag delta -1 |
| 41..43 | Raw operations: put 1, delete 0 |
| 43..53 | Nullable XOR: bitmap mode, validity 0x01, negative-zero bits |
| 53..285 | Leaf/root: one block, four column references |
| 285..397 | Trailer with root reference and checksum |
The test installs these exact bytes and recorded object sizes, opens them through
`SamplesFormat`, and verifies the values and order. It also changes a column
codec to an unknown ID while updating checksums to test that unrequested columns
can be skipped and requested unknown codecs fail explicitly. Legacy catalog
identifier coverage remains in `series_index::catalog` tests.
Do not regenerate this fixture just to make a reader change pass. Preserve its
bytes for future reader compatibility; introduce a separate versioned fixture
when the approved format changes. This is a frozen-byte compatibility check,
not a cross-binary upgrade/downgrade test. Older binaries are not promised to
understand the new opt-in `columnar-samples` catalog identifier.
tests/compatibility/wire/mito_columnar_samples/v1-manifest.json
+38
View File
@@ -0,0 +1,38 @@
{
"format": "columnar-samples",
"version": 1,
"region_id": 4294967297,
"source_file_id": "11111111-1111-1111-1111-111111111111",
"generation": "22222222-2222-2222-2222-222222222222",
"timestamp": {
"column_id": 3,
"name": "ts",
"unit": "millisecond"
},
"fields": [
{
"column_id": 2,
"name": "field_0",
"type": "float64",
"nullable": true
}
],
"files": [
{
"part_id": 0,
"file_size": 397,
"num_rows": 2,
"num_blocks": 1,
"first": {
"table_id": 7,
"tsid": 42
},
"last": {
"table_id": 7,
"tsid": 42
},
"min_timestamp": 100,
"max_timestamp": 100
}
]
}
tests/compatibility/wire/mito_columnar_samples/v1-part.hex
+13
View File
@@ -0,0 +1,13 @@
475453414d504c45010000000000000064000000000000000000000000000000
0900000000000000010100010100000000000000800000000001000000000000
00070000002a0000000000000064000000000000006400000000000000000000
0000000000020000000400000001000000030000001000000000000000100000
00100000005ec3a3cc10ea641301000000000000000000000002000000000000
00200000000000000009000000090000003b1e9c01038b57ca04000000000000
0000000000030000000000000029000000000000000200000002000000f1577f
bf1d628fdd07000000000000000000000004000000020000002b000000000000
000a0000000a0000008e2dc4e1ef3f8b09050000000000000001000000070000
002a00000000000000070000002a000000000000006400000000000000640000
00000000003500000000000000e8000000e800000077c4ce1b4e0f6c87010000
000000000001000000010000008d010000000000003500000000000000475453
414d504c45f202e0cce4a9fdb1