#9477 pinned the old behavior where the HTTP response writer stamped
every row with __name__=<table name>. Under this PR's identity
lifecycle, value-changing functions such as avg_over_time drop the
metric-name identity, so the response carries job only. Also take the
upstream UnionExec snapshot drift (#9491) into the regenerated
tql-cte result.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
#9477 introduced a root path for instant queries whose root is a range
selector, but it bypassed the marked identity column the response layer
reads. Save the table name right after matcher preprocessing (before
setup_context resets it for nonexistent metrics) and attach the
metric-name column on both return paths, mirroring the vector and
matrix selector roots. Without this, GET /v1/query?query=x[5m]
silently dropped __name__ from the response.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Regenerate the standalone and distributed regressions on main d640ab8bc5. Update the distributed TSID binary scan mode to legacy and record the conservative dropped-name uniqueness check in the histogram plan. Visible samples and tabular result schemas are unchanged.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
A table whose own schema carries a column spelled __name__ cannot be
interpreted as PromQL input: the semantic metric name is the table's
identity, so a second, ordinary business column of that name would be
ambiguous. setup_context() now refuses such tables with
ReservedMetricNameColumn (InvalidArguments) at the single selector entry,
before any context is filled; legitimately rewritten names from
label_join/label_replace/count_values are produced above the selector and
never pass through it.
With the reserved-name contract, an unmarked physical __name__ column can
no longer reach the Prometheus HTTP serializer, so the exported_name
auto-rename branch in ColumnLayout::infer is removed and tag names are
emitted verbatim.
Also narrow column_reference_is_constant to an explicit whitelist of
value-preserving plan nodes instead of any single-input passthrough, so a
future extension cannot silently waive the dropped-name uniqueness check.
Tests that constructed a physical __name__ tag as legal input now assert
the entry refusal; the internal __promql_metric_name collision tests are
unaffected.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Replace the UnionDistinctOn shape check in drop_metric_name with a
provable constancy analysis of the identity column. A name that is
rewritten by label_join/label_replace, generated by count_values, or
copied across a binary/set operation is not constant, so dropping it
asserts a unique label set per evaluation timestamp; a selector-attached
literal name is constant and adds no check. The analysis only follows a
column through Projection/Alias/Column and single-holding-inner-join
inputs and conservatively assumes non-constant everywhere else, so it can
over-check but never misses a collision such as
abs(label_join(label_join(m,"__name__","","k"),"k","")).
count_values with a semantic __name__ destination is now a controlled
rename instead of a silent ordinary label: it takes over or attaches the
marked identity with the metric-name role, groups by the formatted sample
value unless a without() modifier excludes it (in which case the label is
neither generated nor grouped), and keeps ordinary destinations ordinary.
The marker-relocation helper is shared between the label functions and
the aggregation path so an ordinary destination that occupies the marked
column's physical name moves the identity aside before grouping keys are
resolved.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
- count_values never forges the identity: the label it generates is always
an ordinary column named as asked, with no metric-name role metadata and
no reuse of the marked column's physical name. A plain aggregate argument
can no longer inject a sample value into the privileged identity column
(which the HTTP layer would otherwise emit as __name__ and TQL would
silently strip). With no marker forged, dropping the name of a
count_values result cannot collide, so the union-subtree check remains
the only duplicate-label-set source.
- binary_match_label_mask no longer lets an ordinary on()/ignoring() label
hit the marked identity by its physical name; only the semantic __name__
resolves to it, and a real user column sharing the name still matches.
- label_join/label_replace move the marked identity to a free internal
name before writing an ordinary destination that occupies its physical
name, so writing a label can no longer silently replace the identity.
- sort_by_label reads a missing or NULL string label as the empty value,
so a series that lost its name sorts first ascending like Prometheus,
instead of NULLS LAST.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Converge the ordinary-label and __name__ paths that had forked during the
rebase:
- resolve_label_col() is the single label-to-column entry: __name__ reads
the marked identity column, ordinary labels resolve like before but can
no longer hit the marked column by its physical name, and a missing
label is absent for every caller (label functions, aggregation, sort).
- label_replace keeps one algorithm for every destination: read source
(missing as ""), match anchored regex, expand captures (empty -> NULL),
keep the old destination value when unmatched. Ordinary destinations may
overwrite existing labels again (#9385 semantics); collapsing series is
refused by the existing runtime label-set check instead of a plan-time
InvalidDestinationLabelName.
- write_destination_label() is the single destination-column entry for
label_replace and label_join, so writing an empty label has one
representation and a zero-source label_join on __name__ removes the
identity instead of projecting an unmarked physical __name__ column.
- drop_metric_name() now asserts a unique label set per evaluation
timestamp when dropping the identity of a union that may ignore it
(abs(a or on(__name__) b)), reusing assert_unique_match_group; TQL
strip_metric_name_column() stays free of PromQL series checks.
- sort_by_label(..., "__name__") on input without a name sorts by a
missing label instead of failing with field_not_found.
Production code is a net reduction (-38 lines); the removed ordinary-label
regex branch, the overwrite tri-state and the duplicated destination
construction are replaced by the three shared entries.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Refresh the planner test expectations after rebasing onto upstream main
(78a7b93292, #9385): ordinary label_join of a missing label now joins as
the empty string and keeps its separator, label_replace of an existing
destination is refused at plan time, and the internal metric-name column
appears in the affected function/subquery plans. Also resolve two
rebase-introduced planner nits: rewrite a clippy filter_map_bool_then and
reformat a regex error construction.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Preserve identity for sorting and last_over_time, drop it after other function results and histogram folding, and resolve semantic name sort keys exclusively through metadata. Seven focused tests pass on Rust 1.96.1. Remaining aggregation, binary/set and output-boundary activation are separate work.
Signed-off-by: Greptime Agent <agent@localhost>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Collect retained projection expression dependencies before pruning instant inputs. Three unmarked regressions failed with missing aux before the seven-line fix and pass afterward, alongside three selector identity tests on Rust 1.96.1.
Signed-off-by: Greptime Agent <agent@localhost>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Attach field-marked name columns at selector lowering and remove identity on unary minus. Raw selector execution and collision tests passed; unary fixture lookup corrected but rerun pending. Intermediate stack checkpoint: remaining operator lifecycle and tabular output boundaries must be completed before activation is merge-ready.
Signed-off-by: Greptime Agent <agent@localhost>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(promql): return range-vector results for instant queries
An instant query of a range selector or subquery (e.g. `up[5m]`) returned
the internal range arrays, which panicked when the frontend decoded them as
dictionaries. Plan such queries as the selected samples with their own
timestamps, and reject range-vector expressions in range queries like
Prometheus does.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): keep encode_substrait on a valid range query
A range query of a range-vector expression now fails like in Prometheus.
Wrap the subquery in max_over_time so the case still encodes both
RangeManipulate nodes through substrait.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): plan wrapped range-vector roots and reject a zero subquery step
TQL EXPLAIN, ANALYZE and AS alias wrap the root in an extension, which the
range-vector path did not unwrap. A zero subquery step divided by zero.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): cover wrapped range-vector roots and a zero subquery step
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): default subquery step and reject range vectors in range queries
A root subquery without a step uses the one-minute default instead of
the query step. The HTTP range query endpoint rejects range-vector
expressions even when start equals end.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: cover root subquery default step
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(promql): align root subqueries with subquery_child_window
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): return millisecond timestamps for range-vector results
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(promql): keep the time index timezone and name its table in errors
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(datanode): execute nested MergeScan plans on the datanode
Give a datanode the ability to decode and execute a plan whose MergeScan
carries a nested plan (e.g. an inner MergeScan over a build table), so a
distributed sub-plan pushed to a datanode can itself fan out to other
datanodes. This is the execution/codec capability only; it is not wired
to any user-facing switch and produces no plan by itself.
Codec (src/query/src/query_engine/default_serializer.rs):
- MergeScanAwareSerializer keeps the request's SessionState and the
engine catalog manager, and decodes a MergeScan payload's tables by
(catalog, schema, table) name through the engine catalog rather than
the region-aware request catalog list (which binds every table to the
request's region and breaks on mismatched columns).
Datanode (src/datanode):
- New DatanodeRegionQueryHandler (region_query.rs) serves region queries
for the MergeScan nodes of a received plan, resolving region leaders
via PartitionRuleManager and dispatching to peer datanodes.
- Wire KvBackendCatalogManager + PartitionRuleManager + NodeClients into
the datanode's catalog/cache registry (with_dist_planner stays false:
a datanode never rewrites its own plan).
Bug fixes uncovered while wiring this up:
- cache registry: register the table/table_info/table_name/table_route/
partition_info caches the datanode's catalog manager and partition
rule manager require (previously missing -> panic on lookup).
- default_serializer: decode_sub_plan bridge no longer drives a future
on a current-thread runtime from within another executor (spawn a
dedicated thread + runtime when not on a multi-thread runtime).
Tested by query_engine::default_serializer unit tests: nested MergeScan
round-trip and engine-catalog resolution with mismatched column names.
No user-facing behavior change.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): keep GreptimeDB function bindings after rebuilding decode state
SessionStateBuilder::new_from_existing collects the function registry
(HashMap) into a Vec and build() re-registers each entry; register_udf
registers aliases before the function's own name and the Vec order is
unstable. Rebuilding the session state to install MergeScanAwareSerializer
after the GreptimeDB functions were registered could therefore rebind a
name/alias (e.g. date_format, an alias of DataFusion's to_char) back to
the built-in function. This path runs for every decoded plan, not only
nested MergeScan ones.
Extract the GreptimeDB function registration into register_greptime_functions
and re-apply it after the final build of both the top-level decode state and
the MergeScan payload state. Add regression tests asserting date_format stays
bound to the GreptimeDB implementation on both paths.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(datanode): address review on nested MergeScan capability
Three follow-ups from review of the nested MergeScan execution capability:
1. Layer the datanode caches (P2b). table_cache derives from
table_info_cache + table_name_cache, and partition_info_cache derives
from table_route_cache, but they were all registered in a single flat
CacheRegistry whose invalidate() runs concurrently with no ordering.
A derived cache could be invalidated and refilled from a base cache
that had not been invalidated yet, leaving stale metadata. Build the
base caches and the derived caches as separate layers of a
LayeredCacheRegistry (reusing the existing layering mechanism) so base
caches finish invalidating before derived ones. Both the production
path (cmd) and the test scaffolding now build the same layered
registry; the flat build_datanode_cache_registry is removed.
2. Don't let a nested MergeScan's internal region query compete for the
concurrency permit (P2a). An outer query holds its permit until its
stream is fully consumed, and a nested MergeScan makes the datanode
issue region queries to peer datanodes as execution stages of that
same query. If such an inner stage also took a permit, the outer query
waiting on it would deadlock against its own held permit (or time out
when max_concurrent_queries > 0). Mark inner DN->DN region queries as
internal stages: QueryRequest gains an flag carried over
the wire in the request header's query_context extensions (reserved
key, not injectable via hints), and the region server skips acquiring
a concurrency permit for internal-stage requests on both acquire
sites. Outer queries still acquire normally.
3. Add an end-to-end capability test
(tests-integration/.../nested_merge_scan_capability_test.rs): hand-build
Join(<local probe region>, MergeScan(build regions on both datanodes))
with mismatched column names and duplicate build keys, ship it to a
datanode over a real TCP Flight endpoint, consume the stream fully and
compare the row multiset against the frontend's reference join result,
assert a real cross-datanode DoGet via an RPC counter, and assert an
inner region failure fails the whole query. Also migrate the
with_real_datanode_grpc_addr + DatanodeRpcStats test scaffolding.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): adapt date_format regression test to the updated DataFusion fork
ScalarUDFImpl's trait bounds changed so its impl can no longer be
downcast via as_any(); assert the function equals the GreptimeDB
implementation instead, which to_char can never satisfy.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(datanode): cover the internal stage under the concurrency limit
Add an end-to-end regression test for the nested MergeScan capability
under max_concurrent_queries = 1: both datanodes' only permit is held by
the outer queries' streams while their inner MergeScans fetch the build
regions from the same datanodes over real Flight connections. The result
consumption is bounded by a timeout, so a broken link anywhere in the
chain (marking the inner request internal, carrying the marker in the
request header, honoring it at the receiving acquire sites) fails the
test instead of deadlocking.
Also add a minimal with_datanode_options_override hook to the cluster
builder, and fix the test module doc: the outer request is handed to
RegionServer::handle_remote_read in process; only the inner region
queries go over the wire.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(datanode): cover cold route loads under unrelated invalidations
The table route cache uses InitStrategy::VersionChecked, whose version
counter belongs to the whole CacheContainer rather than to a single
table: any TableId invalidation bumps the shared version, so a cold
load of one table's route retries when an unrelated table is
invalidated concurrently. Document this as a known limitation on the
region query handler, and pin the behavior with an integration test:
run the nested plans while a background task invalidates an unrelated
table id on every datanode, assert the results still match the expected
multiset (retries only add work, they never return a stale route), and
report the observed load amplification via the cache miss counter
(2 loads without invalidations, 8 over three runs with 24 unrelated
invalidations in this run). Also expose the datanode cache registries
on the test cluster so a test can drive the local invalidation path
that the heartbeat handler drives after a metasrv broadcast.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* docs(query): trim nested plan decoder commentary
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor: remove redundant nested scan support plumbing
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): decode nested MergeScan payloads asynchronously
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(query): avoid unnecessary extension name allocation
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): retain extension decoding for manual session states
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(datanode): keep nested scan query admission unchanged
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(datanode): honor query handler factories and trim nested scan fixtures
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(datanode): retain required cache error import
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(query): reuse registered state for nested scan decoding
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(query): verify payload information schema table functions
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(query): retain nested scan runtime and cold-route guards
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* refactor(query): address nested scan review feedback
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(session): preserve read preference across region query hops
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(session): verify strict context rejection at Flow ingress
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(query): remove redundant serializer error case
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(operator): unify unknown flow option rejection across OSS and enterprise builds
Remove the enterprise-gated passthrough that let unknown flow options
through SQL-layer validation. Both OSS and enterprise binaries now reject
unknown flow options with a single 1004(InvalidArguments) error at the
SQL layer, producing identical sqlness output.
The metasrv-side validate_flow_options remains as a defensive check.
Signed-off-by: discord9 <discord9@163.com>
* fix(flow): use DISTINCT output columns as sink key for streaming flows
When a streaming flow's source table has no primary key, sink key
inference produced an empty key. For SELECT DISTINCT v, the DISTINCT
establishes v as the output key, but the inference only looked at
source primary keys. Combined with the zero-valued placeholder
timestamp, all output rows shared the same storage identity and were
deduplicated into a single row.
Use all non-time-index output columns as the sink key when the plan
is a DISTINCT and no source primary key was inherited.
Signed-off-by: discord9 <discord9@163.com>
* fix(flow): preserve extension options and complete DISTINCT sink identity
Keep enterprise CREATE FLOW options available to the existing handler and report unknown ordinary options as InvalidArguments with the OSS SQL error text. Infer DISTINCT sink keys from every selected non-time-index output column, including when source key columns survive. Wait for asynchronous mirrors in show_create_flow and exercise 1000 distinct sink rows in the SQL regression.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(flow): align meta unknown option status assertions
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(flow): use valid aliases in DISTINCT sink regression SQL
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test(flow): record complete DISTINCT sink SQL results
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(promql): accept evaluation times before the Unix epoch
Prometheus accepts negative timestamps in query parameters; GreptimeDB
rejected them while parsing and would panic converting them in the planner.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: update flow_tql error for a negative TQL end time
A negative end now parses and fails the end >= start check instead of
the timestamp conversion.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): reject evaluation times outside the millisecond range
The signed millisecond conversion cast u128 to i64 unchecked, so a far-past
time wrapped into the future. Reuse the checked conversion from
label_values and return an error instead.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: drop an unused import
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): round query times to milliseconds like Prometheus
Numeric times round the fraction of a second to milliseconds, and
pre-epoch times floor sub-millisecond parts, matching Prometheus'
parseTime and timestamp.FromTime.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): round the float fraction before converting to nanoseconds
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: skip the overflow case where SystemTime cannot hold it
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* perf(promql): zero-copy timestamp payload for millisecond zero-offset ranges
RangeManipulate rebuilt the timestamp range payload per sample through
wide-precision i128 conversion even for millisecond input with zero
offset, where the input is already millisecond and nothing changes.
That turned a previously zero-copy path into a per-sample loop plus a
Vec allocation, regressing range-function queries.
Reuse the input timestamp array directly when the time index is already
millisecond and the offset is zero. Other units and non-zero offsets
keep the wide-precision conversion (required so ns/us data near the
native type limits stays correct under large offsets).
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>
* perf(promql): vectorize timestamp payload with plain i64
RangeManipulate rebuilt the timestamp range payload per sample through
wide-precision i128 conversion, turning a hot path into a scalar loop
with a Vec allocation. Realistic timestamps and offsets stay far inside
i64, so the wide path is unnecessary.
- millisecond + zero offset reuses the input array with zero copy;
- other units / offsets shift in one vectorized i64 pass (no i128, no
per-sample Option, no overflow branch).
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>
* perf(promql): check timestamp payload overflow via arrow try_unary
Replace the silent wrapping shift with try_unary + checked_mul/checked_add.
checked arithmetic lowers to a per-lane overflow flag so the single pass
still vectorizes over the dense i64 buffer; any overflow short-circuits to
an error instead of silently wrapping to a wrong timestamp. No i128, no
per-sample Option.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>
* perf(promql): vectorize cursor scans with i64 fast path
RangeManipulate::calculate_range and InstantManipulate::manipulate
compared every sample through i128 wide-precision arithmetic. For
realistic timestamps and offsets the shifted values stay far inside
i64, so the wide path was pure overhead.
Split both scans into an i64 fast path (checked arithmetic precomputed
once per batch) and an i128 exact fallback. Any input that would
overflow i64 falls back to the i128 scan, keeping window membership
semantics identical.
Verified: 47/47 range_manipulate + instant_manipulate tests, 275/275
promql crate tests, 1612 randomized cases byte-identical to i128 scan.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>
* refactor(promql): generalize cursor scans and share endpoint precheck
Address review comments on PR #9417:
- Generalize scan_ranges/scan_instants over Nanoseconds (i64/i128),
eliminating the duplicated scan bodies.
- Extract shared narrow_scan precheck used by both RangeManipulate and
InstantManipulate, with identical checked endpoint logic.
- Trim doc comments to one line; fix factual errors about fallback
trigger (2262 for ms/s, not native type limits).
Semantics unchanged: same cursor algorithm, same overflow fallback,
same i64-vs-i128 selection, same stale/null/tie handling.
Verified: 47/47 range_manipulate + instant_manipulate tests, clippy clean.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>
* perf(promql): vectorize payload with try_unary and add scan carrier bench
Replace the per-sample i128 payload loop with a single arrow try_unary
pass: i128 intermediate holds any native timestamp shift, only the final
millisecond result must fit i64. This is semantically equivalent to the
original i128 implementation (no silent wrap, no per-sample branching).
Add cursor_scan_carriers A/B benchmark: i64 fast path vs i128 exact
fallback on four workloads. Add five deterministic differential tests
pinning scan_ranges::<i64> and scan_ranges::<i128> to be pointwise
equivalent on boundary shapes (sub-ms samples, fallback triggers,
nanosecond extremes, seeded precision matrix).
Bench (N=4096, 15s cadence):
- payload wall_time 20x15s: 74.94us -> 48.98us (-35%)
- payload wall_time 240x15s: 102.44us
- scan i64 fast path: 10.97us vs i128 19.84us (1.81x)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(promql): restore zero-copy payload for ms+offset=0 and fix comment
Restore the zero-copy fast path for millisecond timestamps with zero
offset, guarded by the input column's exact data type (Timestamp
(Millisecond, None)) rather than self.time_unit. The old time_unit guard
would accept a tz-carrying millisecond column and produce a dictionary
value type disagreeing with the output schema; the datatype guard is the
correct check.
Also fix the comment: arrow's try_unary is a fallible kernel that LLVM
cannot auto-vectorize; the win over the hand-written loop comes from
eliminating the per-sample Option/Vec and branches, not SIMD.
Add a regression test with a timezone-carrying millisecond index to
prove it falls back to the try_unary path.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(promql): replace try_unary with unary + Cell overflow flag
Arrow's try_unary is a fallible kernel that LLVM cannot auto-vectorize.
Replace it with the infallible unary kernel: the closure is a plain
i64 -> i64 mapping that sets a Cell<bool> on overflow instead of
returning Err per sample. The flag is checked once after the loop.
This removes the per-lane Result construction and lets LLVM vectorize
the shift+truncate sequence. Bench (N=4096, 15s cadence):
- window4x15s: 42.12us -> 28.77us (-32%)
- window20x15s: 48.98us -> 27.68us (-43%)
- window240x15s: 102.44us -> 27.61us (-73%)
- regular_sparse: 90.04us -> 29.20us (-68%)
All cases now run at ~27-29us regardless of window size; the zero-copy
path dominates for ms+offset=0.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* perf(promql): dispatch payload conversion on null_count; drop wall-time bench
Null-free batches (dominant PromQL case) take arrow's vectorizable unary
kernel with a Cell<bool> overflow flag; nullable batches take try_unary so
a NULL slot's arbitrary underlying value cannot raise a false batch-wide
overflow error. Add a regression test exercising both paths.
Remove bench_range_manipulate_wall_time (all-ms zero-offset cases only
exercised the zero-copy path, measuring nothing about the conversion) and
its orphaned helpers; restore the cursor-scan bench helpers it shared.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(promql): keep nullable timestamp cursor scans on i128
NULL slots may contain unordered arbitrary values, so the endpoint-only
narrow-scan proof cannot bound their interior timestamp arithmetic. Both
range and instant scans now attempt i64 only for null-free timestamp arrays
and otherwise retain the existing exact i128 path.
Add focused interior-NULL regressions through both active entrypoints,
checking concrete selections and output against the existing wide scan.
The null-free fast path and timestamp payload conversion are unchanged.
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: discord9 <discord9@163.com>
* fix(promql): keep metric name for comparison operators
`collect_metric_names` dropped the response `__name__` for every binary
operator except `and` / `or` / `unless`, so comparisons such as
`node_load1 > 5` came back without the metric name. Grafana rendered them
as `{instance="..."}` and alert templates reading `$labels.__name__` got
nothing.
Follow Prometheus' `shouldDropMetricName` / `resultMetric`: arithmetic
operators drop the name, comparisons keep the left-hand name, except when
the comparison returns a bool or is a one-to-one `on(...)` match. A
`group_left` / `group_right` modifier does not reduce the labels, so the
name is kept there.
Both the HTTP PromQL API and the gRPC gateway share
`retrieve_metric_name_and_result_type`, so one fix covers both.
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* docs: document binary_keeps_metric_name
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* fix(promql): handle on(__name__), scalar lhs and group_right
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* chore(promql): correct result-side docs and drop unreachable String arm
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* fix(promql): drop metric name when explicitly ignored
Handle ignoring(__name__) for one-to-one comparisons and cover
comparison, group_left, group_right and unless behavior.
Signed-off-by: Aumnertic <hznuzc@gmail.com>
---------
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* fix(meta-srv): stop parking the heartbeat response on the region supervisor queue
Signed-off-by: lczllx <2181719471@qq.com>
* test(meta-srv): cover the heartbeat response path at the handler level
Signed-off-by: lczllx <2181719471@qq.com>
---------
Signed-off-by: lczllx <2181719471@qq.com>
The duplicate-label-set rejection must not fire when a label rewrite keeps
every series distinct through another label. Normalizing mixed histogram
boundary representations (`10.0` -> `10`) while `instance` still differs is
the false positive reported in #9444. The check already accounts for it
since it became a per-timestamp row count instead of a planning-time
rejection, but no fixture covered the positive case. Add one, together
with the aggregation variant that produces a genuine duplicate and must
still fail.
Signed-off-by: fengyue-xve <1759519137@qq.com>
* fix(promql): keep vector labels when a computed scalar is on the right
`metric + time()` or `metric * scalar(other)` planned the join with the
scalar side's context, which has no tags, so every series collapsed into
one unlabeled series. Use the vector side's context when the right operand
is scalar-typed.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): update results for labels kept with a computed scalar on the right
v * pi() and v + time() now keep the vector's labels and time index name,
as in Prometheus.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): qualify columns of an unnamed operand in vector joins
A comparison filter's projection has no table name, so after the join its
time index resolved unqualified and was ambiguous when the other operand had
the same column. Also keep the empty-metric branch unchanged for scalar
left operands.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): cover comparison filters joined with a scalar
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Update opensrv-mysql from f3135e5 to 1f3d88c, which replies a proper EOF packet to COM_FIELD_LIST instead of a malformed OK packet that MariaDB 12.3 misparses and hangs on USE information_schema.
Fixes#9158
Signed-off-by: tian1220A <1573612521@qq.com>
* test: support plugins and flownodes in integration cluster builder
Allow the meta-srv mock and the tests-integration distributed cluster
builder to install metasrv Plugins and to start flownodes that register
with the mock metasrv, so integration tests can exercise plugin-provided
DDL handlers and flows against a real multi-datanode cluster.
The default path is unchanged: mock()/mock_with_client_channel_config()
keep their signatures, and GreptimeDbClusterBuilder builds the metasrv
plugin-free and starts no flownode unless with_plugins/with_flownodes is
called.
Signed-off-by: discord9 <discord9@163.com>
* test: narrow cluster plugins to metasrv injection
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* test: remove unnecessary plugin container mutability
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
discord9
discord9
sun
fys
dennis zhuang
shuiyisong
Weny Xu
Dragon Roar
Aum
Niladri Adhikary
fengyue-xve
zhaiAohan