rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-11-14 06:46:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
abc5868073003587b58efebc5cc3caa557fd01aa
greptimedb/Cargo.toml
T
discord9 abc5868073 feat(promql): carry the native plan with serde instead of a hand-written codec 
`native/wire.rs` encoded `NativeTree` by hand: fixed node tags, fixed field
layouts, both directions written out, and a `check_shape` that narrowed the
transport to `Input -> Instant|Window -> Map(Math*)`. The serialization layer
therefore decided which plans could exist. `abs(ceil(x))` compiled but could
not be transported, and every new node kind cost a second codec path plus a
new whitelist entry. Widening the whitelist to nested maps maintained that
interpreter rather than removing it.

The file is deleted. The plan types derive serde and the codec is postcard
1.1.3 -- bincode is unusable here because RUSTSEC-2025-0141 marks the whole
crate unmaintained with no patched version. postcard is maintained, has a
documented stable wire format since 1.0, and writes an f64 as its bit pattern
in eight little-endian bytes, so NaN payloads, signed zero, infinities and
subnormals survive exactly.

What remains hand-written is framing only: magic `NPRQ`, a version, a payload
length. The version is rejected before the payload is deserialized, so an
unknown version is a clean error instead of a silent misparse, and oversize,
trailing bytes and truncation are separate errors. Decoding ends at
`NativeTree::try_new`, so a tree the compiler would not build cannot be
decoded.

Three types keep written impls because they hold invariants. `EvalGrid`
serializes its `GridSpec` and rebuilds through `try_deferred` with an empty
cache -- the grid is never serialized and never materialized on decode.
`SeriesLabels` rebuilds through `try_new`, keeping the sorting and the
duplicate / `__name__`-as-a-tag rejection. `LabelRewrite` travels as its
parameters with the regex recompiled, and fails to decode if the pattern no
longer compiles; compiled internals never travel.

`FusedSeries` now carries its output schema and accepts `PlainRows` and
`partial_fold`, which it used to refuse. A carried schema is only honest if it
is checked, so decode validates every column role, the exact time type and
nullability, missing or extra value columns, and the scalar-root tag rule; a
`SeriesFrame` schema must equal the frame layout the tree derives, and a
partial export must match `state_schema(binding.tags)` including tag order.

Identity stops being display text. `FusedSeries` compared `tree.to_string()`
with a field patch, `AggregateFinalize` hashed the rendering, and
`LabelRewrite` compared its own output; all three are structural now, with
floats compared by `to_bits()` so that signed zero and NaN payloads stay
reflexive under `Eq`. `same_binding` compared `Arc::ptr_eq`, which put an
allocation address into a transportable plan; it compares the ordered fields
and the optional grid spec now, distinguishing raw from evaluated.

Tests assert the properties rather than the byte layout: a non-canonical NaN
payload round-trips by bits, an unknown version is rejected before decode even
when the payload is garbage, non-prefix trees (binary, set, aggregate, scalar,
evaluated input) round-trip, thirteen malformed schemas are each rejected with
their own message, and a multi-chunk chain over a real region is checked
against its exact sample count and contiguous timestamps.

The multi-partition integration test is split in two and labelled honestly. It
used to assert a failure, on the theory that coalescing interleaved multi-chunk
chains; its fixture's grid has one position, so no chain has more than one
chunk and that assertion was already failing at the base commit. One test now
covers the single-position shape and says it does not exercise interleaving;
the other covers a 8192-position grid that does.

Deserialization is bounded by depth, not only by bytes. The derived recursion
walks boxed nodes, and a payload under the 1 MiB limit can nest far deeper than
the stack allows, which aborts the process instead of returning an error. A
thread-local budget guards `NativeNode`'s deserializer, so a hostile payload is
rejected before a complete tree exists. `NativeTree::Deserialize` also routes
through `try_new`, so no path can materialize a tree the constructor would
refuse. Measured under postcard with no `RUST_MIN_STACK` override: 128 nodes
decode, 129 return a decode error. Postcard collapses custom serde errors to
its unit variant, so that rejection does not name the nesting limit; the other
envelope failures (version, size, trailing bytes, truncation) each keep their
own message.

Signed-off-by: discord9 <discord9@163.com>
2026-10-31 01:21:05 +08:00

417 lines
16 KiB
TOML
Raw Blame History

[workspace]
members = [
"src/api",
"src/auth",
"src/cache",
"src/catalog",
"src/cli",
"src/client",
"src/cmd",
"src/common/base",
"src/common/batcher",
"src/common/catalog",
"src/common/config",
"src/common/datasource",
"src/common/decimal",
"src/common/error",
"src/common/event-recorder",
"src/common/frontend",
"src/common/function",
"src/common/greptimedb-telemetry",
"src/common/grpc",
"src/common/grpc-expr",
"src/common/macro",
"src/common/mem-prof",
"src/common/memory-manager",
"src/common/meta",
"src/common/options",
"src/common/plugins",
"src/common/pprof",
"src/common/procedure",
"src/common/procedure-test",
"src/common/query",
"src/common/recordbatch",
"src/common/runtime",
"src/common/session",
"src/common/sql",
"src/common/stat",
"src/common/substrait",
"src/common/telemetry",
"src/common/test-util",
"src/common/time",
"src/common/version",
"src/common/wal",
"src/common/workload",
"src/datanode",
"src/datatypes",
"src/file-engine",
"src/flow",
"src/frontend",
"src/index",
"src/log-query",
"src/log-store",
"src/meta-client",
"src/meta-srv",
"src/metric-engine",
"src/mito-codec",
"src/mito2",
"src/object-store",
"src/operator",
"src/partition",
"src/pipeline",
"src/plugins",
"src/promql",
"src/puffin",
"src/query",
"src/standalone",
"src/servers",
"src/session",
"src/sql",
"src/store-api",
"src/table",
"tests-fuzz",
"tests-integration",
"tests/runner",
]
resolver = "2"
[workspace.package]
version = "1.4.0-alpha.0"
edition = "2024"
license = "Apache-2.0"
[workspace.lints]
clippy.print_stdout = "warn"
clippy.print_stderr = "warn"
clippy.dbg_macro = "warn"
clippy.implicit_clone = "warn"
clippy.result_large_err = "allow"
clippy.large_enum_variant = "allow"
clippy.doc_overindented_list_items = "allow"
clippy.uninlined_format_args = "allow"
rust.unknown_lints = "deny"
rust.unexpected_cfgs = { level = "warn", check-cfg = ['cfg(tokio_unstable)'] }
[workspace.dependencies]
postcard = { version = "1.1.3", default-features = false, features = ["use-std"] }
# DO_NOT_REMOVE_THIS: BEGIN_OF_EXTERNAL_DEPENDENCIES
# We turn off default-features for some dependencies here so the workspaces which inherit them can
# selectively turn them on if needed, since we can override default-features = true (from false)
# for the inherited dependency but cannot do the reverse (override from true to false).
#
# See for more detaiils: https://github.com/rust-lang/cargo/issues/11329
ahash = { version = "0.8", features = ["compile-time-rng"] }
aquamarine = "0.6"
arrow = { version = "=59.2.0", features = ["prettyprint"] }
arrow-array = { version = "=59.2.0", default-features = false, features = ["chrono-tz"] }
arrow-buffer = "=59.2.0"
arrow-flight = "=59.2.0"
arrow-ipc = { version = "=59.2.0", default-features = false, features = ["lz4", "zstd"] }
arrow-schema = { version = "=59.2.0", features = ["serde"] }
arrow-select = "=59.2.0"
async-stream = "0.3"
async-trait = "0.1"
# Remember to update axum-extra, axum-macros when updating axum
axum = "0.8"
axum-extra = "0.10"
axum-macros = "0.5"
backon = "1"
base64 = "0.22"
bigdecimal = "0.4.2"
bitflags = "2.4.1"
bytemuck = "1.12"
bytes = { version = "1.11", features = ["serde"] }
cap-std = "4.0"
chrono = { version = "0.4", features = ["serde"] }
chrono-tz = { version = "0.10", features = ["case-insensitive"] }
clap = { version = "4.4", features = ["derive"] }
config = "0.13.0"
const_format = "0.2"
crc32fast = "1"
criterion = "0.7"
crossbeam-utils = "0.8"
dashmap = "6.1"
datafusion = "=55.1.0"
datafusion-common = "=55.1.0"
datafusion-datasource = "=55.1.0"
datafusion-expr = "=55.1.0"
datafusion-expr-common = "=55.1.0"
datafusion-functions = "=55.1.0"
datafusion-functions-aggregate-common = "=55.1.0"
datafusion-functions-window-common = "=55.1.0"
datafusion-optimizer = "=55.1.0"
datafusion-orc = "0.10.0"
datafusion-pg-catalog = "0.18.3"
datafusion-physical-expr = "=55.1.0"
datafusion-physical-plan = "=55.1.0"
datafusion-proto = "=55.1.0"
datafusion-sql = "=55.1.0"
datafusion-substrait = "=55.1.0"
datafusion_object_store = { package = "object_store", version = "0.13.2" }
deadpool = "0.12"
deadpool-postgres = "0.14"
derive_builder = "0.20"
derive_more = { version = "2.1", features = ["full"] }
dotenv = "0.15"
either = "1.15"
etcd-client = { version = "0.17", features = [
"tls",
"tls-roots",
] }
fs2 = "0.4"
fst = "0.4.7"
futures = "0.3"
futures-util = "0.3"
greptime-proto = { git = "https://github.com/GreptimeTeam/greptime-proto.git", rev = "9533078d6fd4bdaddbca0750cb9bf7ad64b2e568" }
hdfs-native = "0.14.6"
hex = "0.4"
hostname = "0.4.0"
http = "1"
humantime = "2.1"
humantime-serde = "1.1"
hyper = "1.1"
hyper-util = "0.1"
icu_properties = "2.0.1"
indicatif = "0.17"
itertools = "0.14"
jsonb = { git = "https://github.com/GreptimeTeam/jsonb", rev = "dd1e2341e3df35e9de38959328caa7628e81b6b1", default-features = false, features = [
"databend",
] }
lazy_static = "1.4"
local-ip-address = "0.6"
loki-proto = { git = "https://github.com/GreptimeTeam/loki-proto.git", rev = "f69c8924c4babe516373e26a4118be82d976629c" }
meter-core = { git = "https://github.com/GreptimeTeam/greptime-meter.git", rev = "f743c3a5fe8c1f57363eac5866c356b7d91f9b39" }
mockall = "0.13"
moka = "0.12"
nalgebra = "0.33"
nix = { version = "0.30.1", default-features = false, features = ["event", "fs", "process"] }
notify = "8.0"
num_cpus = "1.16"
# Keep object_store 0.13 compatibility while using OpenDAL 0.59.
# TODO(#9240): Switch back to the official bridge after the DataFusion/Arrow/Parquet
# upgrade aligns object_store versions. Keep the bridge and OpenDAL releases compatible.
# https://github.com/GreptimeTeam/greptimedb/issues/9240
object_store_opendal = { git = "https://github.com/GreptimeTeam/opendal.git", rev = "c03e56802b47de3ccfc8fe028e59a459b278a0d2" }
once_cell = "1.18"
opentelemetry-proto = { version = "0.32", features = [
"gen-tonic",
"metrics",
"trace",
"with-serde",
"logs",
] }
ordered-float = { version = "4.3", features = ["serde"] }
otel-arrow-rust = { git = "https://github.com/GreptimeTeam/otel-arrow", rev = "5da284414e9b14f678344b51e5292229e4b5f8d2", features = [
"server",
] }
parking_lot = "0.12"
parquet = { version = "=59.2.0", default-features = false, features = ["arrow", "async", "object_store"] }
parquet-variant = "=59.2.0"
parquet-variant-compute = "=59.2.0"
parquet-variant-json = "=59.2.0"
paste = "1.0"
percent-encoding = "2.3"
pin-project = "1.0"
pretty_assertions = "1.4.0"
prometheus = { version = "0.14", features = ["process"] }
promql-parser = { version = "0.10.0", features = ["ser"] }
prost = { version = "0.14", features = ["no-recursion-limit"] }
prost-types = "0.14"
protoc-bin-vendored = "3.2"
protoc-rust = "2.28"
raft-engine = { version = "0.4.1", default-features = false }
rand = "0.9"
ratelimit = "0.10"
regex = "1.12"
regex-automata = "0.4"
reqwest = { version = "0.13", default-features = false, features = [
"form",
"json",
"query",
"rustls",
"stream",
"multipart",
] }
rskafka = { git = "https://github.com/influxdata/rskafka.git", rev = "80f42eb15eb9419579a376bdf4411893a7da2728", features = [
"transport-tls",
] }
rstest = "0.25"
rstest_reuse = "0.7"
rust_decimal = "1.33"
rustls = { version = "0.23.25", default-features = false }
ryu = "1.0"
sea-query = "0.32"
serde = { version = "1.0", features = ["derive"] }
serde_json = { version = "1.0", features = ["float_roundtrip"] }
serde_with = "3"
serde_yaml_ng = "0.10"
simd-json = "0.15"
similar-asserts = "1.6.0"
smallvec = { version = "1", features = ["serde"] }
snafu = "0.8"
sqlparser = { version = "0.62.0", default-features = false, features = ["std", "visitor", "serde"] }
sqlx = { version = "0.8", default-features = false, features = [
"any",
"macros",
"json",
"runtime-tokio",
"tls-rustls-aws-lc-rs",
"rust_decimal",
] }
stringprep = "0.1"
strum = { version = "0.27", features = ["derive"] }
sysinfo = "0.33"
temp-env = "0.3"
tempfile = "3"
tokio = { version = "1.47", features = ["full"] }
tokio-postgres = "0.7.18"
tokio-rustls = { version = "0.26.2", default-features = false }
tokio-stream = "0.1"
tokio-util = { version = "0.7", features = ["io-util", "compat"] }
toml = "0.8.8"
url = "2.3"
# Pin below 0.14.5 to avoid the `connection_timeout_future` "async fn resumed after
# completion" panic (greptimedb#8265 / grpc-rust#2522). Remove once tonic ships a fix.
tonic = { version = "0.14, <0.14.5", features = ["tls-aws-lc", "gzip", "zstd"] }
tonic-prost = "0.14"
tower = "0.5"
tracing = "0.1"
tracing-appender = "0.2"
tracing-opentelemetry = "0.33.0"
tracing-subscriber = { version = "0.3", features = ["env-filter", "json", "fmt"] }
twox-hash = { version = "2.1", default-features = false, features = ["xxhash64"] }
typetag = "0.2"
uddsketch-rs = "0.1.0"
uuid = { version = "1.17", features = ["serde", "v4", "v7", "fast-rng"] }
vrl = "0.33"
vu128 = "1.1"
winner_tree = "0.1.1"
zstd = "0.13"
# DO_NOT_REMOVE_THIS: END_OF_EXTERNAL_DEPENDENCIES
## workspaces members
api = { path = "src/api" }
auth = { path = "src/auth" }
cache = { path = "src/cache" }
catalog = { path = "src/catalog" }
catio = { git = "https://github.com/GreptimeTeam/catio.git", rev = "c75be64e7a58f620f6340c24b5b8d76f131af6aa" }
cli = { path = "src/cli" }
client = { path = "src/client" }
cmd = { path = "src/cmd", default-features = false }
common-base = { path = "src/common/base" }
common-batcher = { path = "src/common/batcher" }
common-catalog = { path = "src/common/catalog" }
common-config = { path = "src/common/config" }
common-datasource = { path = "src/common/datasource" }
common-decimal = { path = "src/common/decimal" }
common-error = { path = "src/common/error" }
common-event-recorder = { path = "src/common/event-recorder" }
common-frontend = { path = "src/common/frontend" }
common-function = { path = "src/common/function" }
common-greptimedb-telemetry = { path = "src/common/greptimedb-telemetry" }
common-grpc = { path = "src/common/grpc" }
common-grpc-expr = { path = "src/common/grpc-expr" }
common-macro = { path = "src/common/macro" }
common-mem-prof = { path = "src/common/mem-prof" }
common-memory-manager = { path = "src/common/memory-manager" }
common-meta = { path = "src/common/meta" }
common-options = { path = "src/common/options" }
common-plugins = { path = "src/common/plugins" }
common-pprof = { path = "src/common/pprof" }
common-procedure = { path = "src/common/procedure" }
common-procedure-test = { path = "src/common/procedure-test" }
common-query = { path = "src/common/query" }
common-recordbatch = { path = "src/common/recordbatch" }
common-runtime = { path = "src/common/runtime" }
common-session = { path = "src/common/session" }
common-sql = { path = "src/common/sql" }
common-stat = { path = "src/common/stat" }
common-telemetry = { path = "src/common/telemetry" }
common-test-util = { path = "src/common/test-util" }
common-time = { path = "src/common/time" }
common-version = { path = "src/common/version" }
common-wal = { path = "src/common/wal" }
common-workload = { path = "src/common/workload" }
datanode = { path = "src/datanode" }
datatypes = { path = "src/datatypes" }
file-engine = { path = "src/file-engine" }
flow = { path = "src/flow" }
frontend = { path = "src/frontend", default-features = false }
index = { path = "src/index" }
log-query = { path = "src/log-query" }
log-store = { path = "src/log-store" }
meta-client = { path = "src/meta-client" }
meta-srv = { path = "src/meta-srv" }
metric-engine = { path = "src/metric-engine" }
mito-codec = { path = "src/mito-codec" }
mito2 = { path = "src/mito2" }
object-store = { path = "src/object-store" }
operator = { path = "src/operator" }
partition = { path = "src/partition" }
pipeline = { path = "src/pipeline" }
plugins = { path = "src/plugins" }
promql = { path = "src/promql" }
puffin = { path = "src/puffin" }
query = { path = "src/query" }
servers = { path = "src/servers" }
session = { path = "src/session" }
sql = { path = "src/sql" }
standalone = { path = "src/standalone" }
store-api = { path = "src/store-api" }
substrait = { path = "src/common/substrait" }
table = { path = "src/table" }
[workspace.dependencies.meter-macros]
git = "https://github.com/GreptimeTeam/greptime-meter.git"
rev = "f743c3a5fe8c1f57363eac5866c356b7d91f9b39"
[patch.crates-io]
substrait = { git = "https://github.com/GreptimeTeam/substrait-rs.git", rev = "91ec978b0649417ad3da8390e7a515baec723b1b" }
datafusion = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-common = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-datasource = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-expr = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-expr-common = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-functions = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-functions-aggregate-common = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-functions-window-common = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-optimizer = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-physical-expr = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-physical-expr-common = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-physical-plan = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-proto = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-sql = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-substrait = { git = "https://github.com/GreptimeTeam/datafusion.git", rev = "e3221b8f74b15c7192621576e5d1e354a6cf097a" }
datafusion-pg-catalog = { git = "https://github.com/datafusion-contrib/datafusion-postgres.git", rev = "ffb14a52b3d7c7489559812e1ec38096f225ce93" }
arrow-pg = { git = "https://github.com/datafusion-contrib/datafusion-postgres.git", rev = "ffb14a52b3d7c7489559812e1ec38096f225ce93" }
sqlparser = { git = "https://github.com/GreptimeTeam/sqlparser-rs.git", rev = "9e9019bb1c7040ed956f654e39378dd42ab17884" }
# Temporary: use the GreptimeTeam fork of tikv-jemalloc-sys embedding
# jemalloc 5.3.1 + backport of 54f22c83 ("Initialize TSD tcache before
# enabling it"), fixing the TSD/tcache enable-before-init reentrancy
# window. Remove once tikv/jemallocator ships a jemalloc snapshot that
# includes the fix. See GreptimeTeam/jemalloc#1.
tikv-jemalloc-sys = { git = "https://github.com/GreptimeTeam/jemallocator", rev = "e254a7eaf13a9ede04953ddefd173b5303018530" }
[profile.release]
debug = 1
[profile.nightly]
inherits = "release"
strip = "debuginfo"
lto = "thin"
debug = false
incremental = false
[profile.ci]
inherits = "dev"
strip = true
[profile.dev.package.sqlness-runner]
debug = false
strip = true
[profile.dev.package.tests-fuzz]
debug = false
strip = true
Reference in New Issue View Git Blame Copy Permalink