Remove plan-global label-set occupancy and the speculative split-series tree guard. Keep the native evaluator, finality, memory budgets, errors and ordinary RecordBatch output local to each scan partition.
Narrow native admission to selectors and the small unary POC subset. Document that cross-partition final-label conflicts and semantic merging are outside this POC, and cover real rate/abs(rate) execution across dense and sparse storage spanning SST and memtable data.
Signed-off-by: discord9 <discord9@163.com>
Keep native queries available for unflushed memtables and SSTs with incomplete columnar series or samples coverage by selecting the ordinary raw-series reader. Preserve structural validation and propagate artifact read errors instead of retrying them through legacy execution.
Cover all four incomplete-readiness states in public PromQL queries, both storage scan modes and ordering, complete prepared-source execution, and artifact error propagation.
Signed-off-by: discord9 <discord9@163.com>
The flow and nested-MergeScan ingress tests hardcoded "follower" among
the read-preference values that must be rejected, coupling them to the
current leader-only ReadPreference enum. Assert only genuinely invalid
values there, matching the wire-values test which already limits itself
to ["", "unknown"]. This keeps the ingress tests valid for builds that
extend ReadPreference with more variants.
Signed-off-by: Ning Sun <sunning@greptime.com>
* refactor(operator): route shared requests through NodeManager
Remove the concrete NodeClients requester API and use the existing node manager for owned and shared requests. Preserve the original write hint while the legacy error adapter remains in place.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(meta): preserve retry policy through error hints
Propagate client errors through External and let DDL read retry hints while preserving cleanup and terminal-child policies. Region transport failures retain the legacy Retryable classification, which now also participates in bounded write retries.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(operator): restore sparse partition tags when rebuilding writes
Decode retained primary keys when refreshed partition expressions need different tags. Keep the partitioner and multi-region sparse write regression tests with the fix.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* fix(operator): stop detached write retries when callers cancel
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat(frontend): configure retries across distributed write paths
Wire the shared dispatcher through frontend and logical batching, preserving partition columns and request headers. Keep standalone retries disabled and include configuration, batching, and storage regression coverage.
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
* feat(jaeger): support v3 HTTP query APIs
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(jaeger): preserve full traces with v3 time hints
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs(jaeger): clarify validation of v3 lookup hints
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(jaeger): bound v3 searches and preserve v1 binary attributes
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(jaeger): match resource attributes in v0 tag filters
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): return range-vector results for instant queries
An instant query of a range selector or subquery (e.g. `up[5m]`) returned
the internal range arrays, which panicked when the frontend decoded them as
dictionaries. Plan such queries as the selected samples with their own
timestamps, and reject range-vector expressions in range queries like
Prometheus does.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): keep encode_substrait on a valid range query
A range query of a range-vector expression now fails like in Prometheus.
Wrap the subquery in max_over_time so the case still encodes both
RangeManipulate nodes through substrait.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): plan wrapped range-vector roots and reject a zero subquery step
TQL EXPLAIN, ANALYZE and AS alias wrap the root in an extension, which the
range-vector path did not unwrap. A zero subquery step divided by zero.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): cover wrapped range-vector roots and a zero subquery step
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): default subquery step and reject range vectors in range queries
A root subquery without a step uses the one-minute default instead of
the query step. The HTTP range query endpoint rejects range-vector
expressions even when start equals end.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: cover root subquery default step
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(promql): align root subqueries with subquery_child_window
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): return millisecond timestamps for range-vector results
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(promql): keep the time index timezone and name its table in errors
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(datanode): execute nested MergeScan plans on the datanode
Give a datanode the ability to decode and execute a plan whose MergeScan
carries a nested plan (e.g. an inner MergeScan over a build table), so a
distributed sub-plan pushed to a datanode can itself fan out to other
datanodes. This is the execution/codec capability only; it is not wired
to any user-facing switch and produces no plan by itself.
Codec (src/query/src/query_engine/default_serializer.rs):
- MergeScanAwareSerializer keeps the request's SessionState and the
engine catalog manager, and decodes a MergeScan payload's tables by
(catalog, schema, table) name through the engine catalog rather than
the region-aware request catalog list (which binds every table to the
request's region and breaks on mismatched columns).
Datanode (src/datanode):
- New DatanodeRegionQueryHandler (region_query.rs) serves region queries
for the MergeScan nodes of a received plan, resolving region leaders
via PartitionRuleManager and dispatching to peer datanodes.
- Wire KvBackendCatalogManager + PartitionRuleManager + NodeClients into
the datanode's catalog/cache registry (with_dist_planner stays false:
a datanode never rewrites its own plan).
Bug fixes uncovered while wiring this up:
- cache registry: register the table/table_info/table_name/table_route/
partition_info caches the datanode's catalog manager and partition
rule manager require (previously missing -> panic on lookup).
- default_serializer: decode_sub_plan bridge no longer drives a future
on a current-thread runtime from within another executor (spawn a
dedicated thread + runtime when not on a multi-thread runtime).
Tested by query_engine::default_serializer unit tests: nested MergeScan
round-trip and engine-catalog resolution with mismatched column names.
No user-facing behavior change.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): keep GreptimeDB function bindings after rebuilding decode state
SessionStateBuilder::new_from_existing collects the function registry
(HashMap) into a Vec and build() re-registers each entry; register_udf
registers aliases before the function's own name and the Vec order is
unstable. Rebuilding the session state to install MergeScanAwareSerializer
after the GreptimeDB functions were registered could therefore rebind a
name/alias (e.g. date_format, an alias of DataFusion's to_char) back to
the built-in function. This path runs for every decoded plan, not only
nested MergeScan ones.
Extract the GreptimeDB function registration into register_greptime_functions
and re-apply it after the final build of both the top-level decode state and
the MergeScan payload state. Add regression tests asserting date_format stays
bound to the GreptimeDB implementation on both paths.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(datanode): address review on nested MergeScan capability
Three follow-ups from review of the nested MergeScan execution capability:
1. Layer the datanode caches (P2b). table_cache derives from
table_info_cache + table_name_cache, and partition_info_cache derives
from table_route_cache, but they were all registered in a single flat
CacheRegistry whose invalidate() runs concurrently with no ordering.
A derived cache could be invalidated and refilled from a base cache
that had not been invalidated yet, leaving stale metadata. Build the
base caches and the derived caches as separate layers of a
LayeredCacheRegistry (reusing the existing layering mechanism) so base
caches finish invalidating before derived ones. Both the production
path (cmd) and the test scaffolding now build the same layered
registry; the flat build_datanode_cache_registry is removed.
2. Don't let a nested MergeScan's internal region query compete for the
concurrency permit (P2a). An outer query holds its permit until its
stream is fully consumed, and a nested MergeScan makes the datanode
issue region queries to peer datanodes as execution stages of that
same query. If such an inner stage also took a permit, the outer query
waiting on it would deadlock against its own held permit (or time out
when max_concurrent_queries > 0). Mark inner DN->DN region queries as
internal stages: QueryRequest gains an flag carried over
the wire in the request header's query_context extensions (reserved
key, not injectable via hints), and the region server skips acquiring
a concurrency permit for internal-stage requests on both acquire
sites. Outer queries still acquire normally.
3. Add an end-to-end capability test
(tests-integration/.../nested_merge_scan_capability_test.rs): hand-build
Join(<local probe region>, MergeScan(build regions on both datanodes))
with mismatched column names and duplicate build keys, ship it to a
datanode over a real TCP Flight endpoint, consume the stream fully and
compare the row multiset against the frontend's reference join result,
assert a real cross-datanode DoGet via an RPC counter, and assert an
inner region failure fails the whole query. Also migrate the
with_real_datanode_grpc_addr + DatanodeRpcStats test scaffolding.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): adapt date_format regression test to the updated DataFusion fork
ScalarUDFImpl's trait bounds changed so its impl can no longer be
downcast via as_any(); assert the function equals the GreptimeDB
implementation instead, which to_char can never satisfy.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(datanode): cover the internal stage under the concurrency limit
Add an end-to-end regression test for the nested MergeScan capability
under max_concurrent_queries = 1: both datanodes' only permit is held by
the outer queries' streams while their inner MergeScans fetch the build
regions from the same datanodes over real Flight connections. The result
consumption is bounded by a timeout, so a broken link anywhere in the
chain (marking the inner request internal, carrying the marker in the
request header, honoring it at the receiving acquire sites) fails the
test instead of deadlocking.
Also add a minimal with_datanode_options_override hook to the cluster
builder, and fix the test module doc: the outer request is handed to
RegionServer::handle_remote_read in process; only the inner region
queries go over the wire.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(datanode): cover cold route loads under unrelated invalidations
The table route cache uses InitStrategy::VersionChecked, whose version
counter belongs to the whole CacheContainer rather than to a single
table: any TableId invalidation bumps the shared version, so a cold
load of one table's route retries when an unrelated table is
invalidated concurrently. Document this as a known limitation on the
region query handler, and pin the behavior with an integration test:
run the nested plans while a background task invalidates an unrelated
table id on every datanode, assert the results still match the expected
multiset (retries only add work, they never return a stale route), and
report the observed load amplification via the cache miss counter
(2 loads without invalidations, 8 over three runs with 24 unrelated
invalidations in this run). Also expose the datanode cache registries
on the test cluster so a test can drive the local invalidation path
that the heartbeat handler drives after a metasrv broadcast.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* docs(query): trim nested plan decoder commentary
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor: remove redundant nested scan support plumbing
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): decode nested MergeScan payloads asynchronously
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(query): avoid unnecessary extension name allocation
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): retain extension decoding for manual session states
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(datanode): keep nested scan query admission unchanged
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(datanode): honor query handler factories and trim nested scan fixtures
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(datanode): retain required cache error import
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(query): reuse registered state for nested scan decoding
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(query): verify payload information schema table functions
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(query): retain nested scan runtime and cold-route guards
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(query): address nested scan review feedback
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(session): preserve read preference across region query hops
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(session): verify strict context rejection at Flow ingress
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(query): remove redundant serializer error case
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test: support plugins and flownodes in integration cluster builder
Allow the meta-srv mock and the tests-integration distributed cluster
builder to install metasrv Plugins and to start flownodes that register
with the mock metasrv, so integration tests can exercise plugin-provided
DDL handlers and flows against a real multi-datanode cluster.
The default path is unchanged: mock()/mock_with_client_channel_config()
keep their signatures, and GreptimeDbClusterBuilder builds the metasrv
plugin-free and starts no flownode unless with_plugins/with_flownodes is
called.
Signed-off-by: discord9 <discord9@163.com>
* test: narrow cluster plugins to metasrv injection
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test: remove unnecessary plugin container mutability
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* feat(log-store): collect object store WAL objects below a persisted reclaim boundary
Every object header carries a reclaim boundary: every object below it was
fully flushed when the object was sealed. The writer computes the boundary
from the catalog and the obsolete watermarks after every obsolete and every
indexed object, and stamps it on every sealed object and on the start
object. The boundary of the last indexed object permits deletion: every
present object below it, indexed, listed by recovery off the chain, or
left by a failed batch, is deleted in the background, at most four at a
time.
Recovery judges every candidate with its own boundary: a link to a
predecessor below it holds whether or not the predecessor is present, and
the chain stops there. A read that listed an object before it was
collected skips it instead of failing.
The header grows from 46 to 54 bytes; the format version stays 1.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* perf(log-store): scan the reclaim boundary from the previous boundary
The boundary was recomputed by scanning every region's indexed objects
from the first one, so the flushed objects of busy regions were scanned
again on every indexed object while another region held the boundary.
Scan the objects in sequence order from the previous boundary instead
and stop at the first one holding a live segment. Watermarks never move
down and a region first appears above the boundary, so the result is
the same and an object is not scanned again once the boundary passed it.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(log-store): name the parked operation helpers after both their uses
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* feat(log-store): skip corrupted object store WAL segments per region
A segment that does not decode, because its checksum does not match or
its content disagrees with its footer entry, is fetched once more. If it
still does not decode, the read follows `on_corrupted_segment`: `skip`
records a WAL hole for the region with the object key, sequence and the
footer's entry id range, increments
`greptime_logstore_object_store_wal_skipped_segments_total`, logs a
warning and continues with the next object; `fail` fails the read. A
failed fetch fails the read in both modes. Holes are kept in memory per
region and can be listed from the store.
The standalone test builder allocates WAL options through
`build_standalone_wal_provider`, and integration tests run a standalone
instance on the object store WAL across restarts, on S3 when the
`GT_S3_*` variables are set. The standalone shutdown runs every step and
returns the first error.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* test(cmd): cover the best-effort standalone stop
Build a standalone instance whose leader services fail to stop and whose
datanode servers fail afterwards, and check that `Instance::stop` still
stops the object store WAL and returns the leader services error.
Rename `skip_segment` to `record_skipped_segment` and the file store
integration test to `test_standalone_object_store_wal_survives_restarts_on_file`.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
---------
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
* feat: add CORS support for gRPC-Web on the frontend gRPC server
A browser-based gRPC-Web client cannot read a cross-origin response. The
request carries a non-simple content type, so the browser sends an
`OPTIONS` preflight first, and it drops the call when the response has no
`Access-Control-Allow-Origin`.
Add `enable_cors` and `cors_allowed_origins` to `[grpc]`, threaded from
`GrpcOptions` to `GrpcServerConfig` the same way `tls` is.
The CORS layer sits outside `tonic_web::GrpcWebLayer`, so it answers the
preflight before the request reaches the gRPC routes. It exposes
`grpc-status`, `grpc-message` and `grpc-status-details-bin`, which a
browser cannot read otherwise. An empty `cors_allowed_origins` allows any
origin.
`enable_cors` defaults to false, unlike `[http]`. `GrpcOptions` is shared
by the public `[grpc]` section and the internal `[internal_grpc]` one, and
serde cannot tell them apart, so a true default would also turn CORS on
for the internal gRPC listeners: the frontend internal gRPC server, and
the datanode and flownode gRPC servers. None of them authenticate callers,
and a browser on the host or in the cluster network can reach all of them.
Set `enable_cors = true` to turn it on.
Tests start a real server on an ephemeral port and cover the preflight, a
custom origin list, and the disabled case.
Update the example TOMLs and regenerate config/config.md.
Signed-off-by: lczllx <2181719471@qq.com>
* fix: drop the redundant OPTIONS from the gRPC CORS allow_methods
Signed-off-by: lczllx <2181719471@qq.com>
* test: assert the allow-headers and allow-methods headers in the gRPC CORS preflight
The preflight answers with `access-control-allow-headers: *` from
`AllowHeaders::any()`, and with `access-control-allow-methods: post` from the
POST-only `allow_methods` list. Pin both, the way the HTTP CORS test pins its
own headers: a missing allow-headers header would let the preflight pass the
origin check and still have the browser block every gRPC-Web call, which a
non-browser client would never notice.
Signed-off-by: lczllx <2181719471@qq.com>
* docs(config): stop the example configs from setting the new gRPC CORS keys
Signed-off-by: lczllx <2181719471@qq.com>
* test(servers): cover the exposed gRPC CORS headers and pin the option plumbing
Signed-off-by: lczllx <2181719471@qq.com>
* docs(config): document the gRPC CORS origin example with `#+`
Signed-off-by: lczllx <2181719471@qq.com>
* fix(frontend): keep CORS off the internal gRPC server
Signed-off-by: lczllx <2181719471@qq.com>
* docs(grpc): document that the internal gRPC server never serves CORS
Signed-off-by: lczllx <2181719471@qq.com>
* Update src/servers/src/grpc.rs
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: trim redundant comments
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor: share CORS origin parsing and simplify gRPC-Web CORS tests
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor: enable gRPC CORS only on the frontend public server
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* docs(config): clarify the gRPC CORS options
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: lczllx <2181719471@qq.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Dennis Zhuang <killme2008@gmail.com>
discord9
discord9
Yingwen
Ning Sun
Weny Xu
jeremyhi
dennis zhuang
shuiyisong
Dragon RoarandDennis Zhuang