mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-11-22 02:34:00 +00:00
8348d3d2d4600ade47dbbfb05fac36e4e5f2d53b
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c9e7f6ca90 |
refactor(promql): carry the fused plan as one derived payload
The node still framed its fields by hand: the grid spec's integers, a 0/1 marker for the metric name, a length prefix before every string, the tag count, a format tag, and separate length-prefixed blobs for the partial fold and the output schema, with a matching Reader on the way in. That is per-field serialization sitting inside a change whose point was to stop writing per-field serialization, and it kept the maintenance cost: any new field touched both directions, and a test located a byte by computing 8 + tree.len() + 8 + 8 + 8. One derived payload replaces it. The envelope is now a version, a payload length, and a single Postcard-encoded `FusedSeriesPayload` holding the tree, the grid spec, the binding's name/time index/tags, the output format, the statement text, the optional partial fold and the output schema. Substrait still carries the children, which are exactly the fields this format should not be inventing. `text`, `Reader` and `MAX_TAG_COUNT` are gone; the tag bound existed only because a hand-written length prefix needs one. Every carried-schema check stays: the tag-order-sensitive comparison against `state_schema(binding.tags)` for a partial export, exact equality with `series_frame_output_schema` for a frame node, and the full column-role validation for plain rows. Decode also requires the payload to be consumed exactly, so a byte appended after a valid payload is rejected rather than ignored -- `postcard::from_bytes` would have accepted it, which is why the sub-payload framing had that hole. Two tests were reading the format rather than the behaviour: The depth-limit test asserted that `RUST_MIN_STACK` was unset, but CI sets it to 8 MiB, so the assertion failed before the decoder ran and the test could never do its job there. It now runs on an explicit 2 MiB worker stack, which is the stack a normal worker has, and asserts from inside that thread that 128 nodes decode and 129 are rejected. The integration test located the embedded version by assuming an offset and comparing raw bytes. The framing change made that offset stale and it failed. It now finds the envelope, decodes the version word and asserts the number, checks the length word against the payload, and prints the observed bytes when either assertion trips. Signed-off-by: discord9 <discord9@163.com> |
||
|
|
abc5868073 |
feat(promql): carry the native plan with serde instead of a hand-written codec
`native/wire.rs` encoded `NativeTree` by hand: fixed node tags, fixed field layouts, both directions written out, and a `check_shape` that narrowed the transport to `Input -> Instant|Window -> Map(Math*)`. The serialization layer therefore decided which plans could exist. `abs(ceil(x))` compiled but could not be transported, and every new node kind cost a second codec path plus a new whitelist entry. Widening the whitelist to nested maps maintained that interpreter rather than removing it. The file is deleted. The plan types derive serde and the codec is postcard 1.1.3 -- bincode is unusable here because RUSTSEC-2025-0141 marks the whole crate unmaintained with no patched version. postcard is maintained, has a documented stable wire format since 1.0, and writes an f64 as its bit pattern in eight little-endian bytes, so NaN payloads, signed zero, infinities and subnormals survive exactly. What remains hand-written is framing only: magic `NPRQ`, a version, a payload length. The version is rejected before the payload is deserialized, so an unknown version is a clean error instead of a silent misparse, and oversize, trailing bytes and truncation are separate errors. Decoding ends at `NativeTree::try_new`, so a tree the compiler would not build cannot be decoded. Three types keep written impls because they hold invariants. `EvalGrid` serializes its `GridSpec` and rebuilds through `try_deferred` with an empty cache -- the grid is never serialized and never materialized on decode. `SeriesLabels` rebuilds through `try_new`, keeping the sorting and the duplicate / `__name__`-as-a-tag rejection. `LabelRewrite` travels as its parameters with the regex recompiled, and fails to decode if the pattern no longer compiles; compiled internals never travel. `FusedSeries` now carries its output schema and accepts `PlainRows` and `partial_fold`, which it used to refuse. A carried schema is only honest if it is checked, so decode validates every column role, the exact time type and nullability, missing or extra value columns, and the scalar-root tag rule; a `SeriesFrame` schema must equal the frame layout the tree derives, and a partial export must match `state_schema(binding.tags)` including tag order. Identity stops being display text. `FusedSeries` compared `tree.to_string()` with a field patch, `AggregateFinalize` hashed the rendering, and `LabelRewrite` compared its own output; all three are structural now, with floats compared by `to_bits()` so that signed zero and NaN payloads stay reflexive under `Eq`. `same_binding` compared `Arc::ptr_eq`, which put an allocation address into a transportable plan; it compares the ordered fields and the optional grid spec now, distinguishing raw from evaluated. Tests assert the properties rather than the byte layout: a non-canonical NaN payload round-trips by bits, an unknown version is rejected before decode even when the payload is garbage, non-prefix trees (binary, set, aggregate, scalar, evaluated input) round-trip, thirteen malformed schemas are each rejected with their own message, and a multi-chunk chain over a real region is checked against its exact sample count and contiguous timestamps. The multi-partition integration test is split in two and labelled honestly. It used to assert a failure, on the theory that coalescing interleaved multi-chunk chains; its fixture's grid has one position, so no chain has more than one chunk and that assertion was already failing at the base commit. One test now covers the single-position shape and says it does not exercise interleaving; the other covers a 8192-position grid that does. Deserialization is bounded by depth, not only by bytes. The derived recursion walks boxed nodes, and a payload under the 1 MiB limit can nest far deeper than the stack allows, which aborts the process instead of returning an error. A thread-local budget guards `NativeNode`'s deserializer, so a hostile payload is rejected before a complete tree exists. `NativeTree::Deserialize` also routes through `try_new`, so no path can materialize a tree the constructor would refuse. Measured under postcard with no `RUST_MIN_STACK` override: 128 nodes decode, 129 return a decode error. Postcard collapses custom serde errors to its unit variant, so that rejection does not name the nesting limit; the other envelope failures (version, size, trailing bytes, truncation) each keep their own message. Signed-off-by: discord9 <discord9@163.com> |
||
|
|
6f99b36e44 |
fix(promql): bound the prefix depth and name what a refusal names
Lower MAX_DEPTH to 32. The old 128 was validated recursively, and a debug build overflows a default test-thread stack well before reaching it, so the depth guard itself could abort instead of refusing. Report the refused node by its own name instead of "unsupported", which is what every kind used to say. Add the multi-partition datanode case. It asserts the failure the review reproduced - a series chain cut by CoalescePartitionsExec no longer decodes - so the limitation stays visible and turns red when the region merge becomes chain-atomic. Reconcile the retained-rate comment with the values the test asserts. Signed-off-by: discord9 <discord9@163.com> |
||
|
|
1d4aae7371 |
fix(promql): make the native prefix codec agree with itself
A payload the encoder accepts has to be one this version can decode. Three places disagreed. An `Input`'s field count was bounded only on the way in, so a 129-field `Input → Instant` encoded to 983 bytes and then failed to decode. The decoder's limit was `MAX_COUNT`, a constant that also bounds nesting depth and `Map` operation count, which is why the missing half was easy to miss; each limit now has its own name and both sides check it. `FusedSeries::serialize` only checked that the tag count fit in a `u32` while `deserialize` bounded it at 1024. Both sides now share `MAX_TAG_COUNT`. `FusedSeries::deserialize` passed an empty schema to `try_new_with_format`, which checks a frame node's declared schema against the layout derived from its tree rather than overwriting it. Every serialized frame node was therefore impossible to rebuild. The schema is derived from the decoded tree now, which is where it comes from in the first place, so nothing is added to the wire. Encoding refuses an over-limit count instead of writing a payload it cannot read back, and a field-count overflow no longer reports "operation count exceeds maximum". `rejects_excess_operation_count` hand-built a payload whose operation count was read from the wrong offset: it placed an `Instant` where the decoder expects the count, so it exercised the limit with 258 while claiming 129. It now starts from a tree the codec accepts, asserts the bytes it edits are the count it means, and checks the error names both the operation limit and the value. The same is added for field count, tag count and nesting. The integration test moves to its own file: it was written against `tests-integration/tests/raw_series.rs`, which the raw-fan-in line creates and this one does not carry. Signed-off-by: discord9 <discord9@163.com> |