* fix(servers): resolve port from IPv6 bind addresses (#9142)
Signed-off-by: immanuwell <pchpr.00@list.ru>
(cherry picked from commit 5ec679fed9)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: register region migration procedures before returning IDs (#9163)
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit c65e8d0962)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(promql): skip empty batches in SeriesDivide stream (#9178)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 5e57b00a95)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(servers): merge duplicate timeseries across RecordBatches in Prometheus remote read (#9032)
* fix(servers): merge timeseries across RecordBatches in Prometheus remote read
Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
* fix(servers): exercise recordbatches_to_timeseries and sort samples with sort_unstable_by_key
---------
Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
(cherry picked from commit 59eb63c663)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(prometheus): honor label matchers in __name__ values query (#9134)
* fix(prometheus): honor label matchers in __name__ values query
`/api/v1/label/__name__/values?match[]={pod="abc"}` dropped every matcher
other than `__name__` and returned all metrics in the schema. No error,
just the wrong list. Grafana's metrics browser sends this request, so
picking a label value there did nothing.
Selectors that only constrain `__name__` keep answering from table
metadata. A selector constraining an ordinary label now goes to the data:
scan each metric engine physical table for distinct `__table_id` in the
time range, map the ids back to metric names, then apply the selector's
own `__name__` matchers.
Only metric engine tables are covered; other engines share no column space
to scan.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(prometheus): batch-resolve metric names by table id
Building a full table-id-to-name map meant walking every table in the
schema and holding all of them in memory, just to name the handful the
scan returned. Use `tables_by_ids` instead — one batch KV read over the
ids the scan actually produced.
The catalog walk stays, but only to find the physical tables to scan.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): read an absent label as the empty string
A matcher on a label the series does not carry only worked when the table
had no column for it at all. Where the column exists but is NULL on that
row -- the norm for logical metrics sharing a metric engine physical
table, which holds the union of their label columns -- three-valued logic
dropped the row, so `host!="host1"` and `host=""` missed every metric
without a host label.
Coalesce nullable string label columns to "" for matchers that accept the
empty string, rather than only for the OTLP temporality marker. Equality
matchers are untouched; they cannot match NULL either way.
This is the Prometheus compatibility fix#8970 deliberately kept out of
its own scope. The cost is visible in the regex sqlness plan: the
predicate becomes a CASE, so the scan loses its LastRow selector and
grows a FilterExec. Only negative and empty-accepting matchers pay it.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): don't panic on pre-epoch label value bounds
`rewrite_label_values_query` unwrapped `duration_since(UNIX_EPOCH)`, which
returns an error for an instant before the epoch. `start=1969-12-31T23:59:59Z`
parses as valid RFC3339, so the request panicked instead of answering.
Recover the sign from the error branch, and report a value beyond i64
milliseconds as an error rather than wrapping the cast.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(prometheus): drop applicable_matchers, share the distinct scan
With the planner reading an absent label as empty, the frontend no longer
needs to pre-filter matchers per physical table. Removing that exposed a
second problem: a physical table that never took a column from a logical
table exposes no `__table_id`, and projecting it failed the whole request.
Skip those tables; the only thing that can miss is a metric with no labels.
Also pulls out the plan-build-execute-collect sequence the two label value
scans had in common.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 007836ba7a)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): cancel cache construction for incomplete scans (#9254)
* fix(mito2): cancel cache construction for incomplete scans
CacheBatchBuffer spawned the background concat task and dropped its join
handle. A scan that was cancelled or failed therefore left the task alive:
it kept compacting already queued batches, and while waiting for a range
result memory permit it held them, even though without a finish command
the result can never be put into the cache.
Keep the handle and abort it when the buffer is dropped. The handle is
cleared once the task owns the finish command, so a completed scan still
populates the cache after its stream is dropped. Abort does not preempt a
concat that is already running; it takes effect the next time the task is
polled.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(mito2): wait for the permit park before cancelling the buffer
An empty buffered_batches only proves the batches were enqueued, so the
cancellation test could abort a concat task that had never been polled.
Count acquisitions that find too few permits, a test-only signal, and drop
the buffer once the task has reached that wait. Nothing awaits between the
check and the parking, so an observed increment means the caller is about
to wait for permits the test holds for the rest of the case.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 43ce07fa69)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: merge every state row in geo_path and json_encode_path (#9339)
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit cc1be82858)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: align ordered aggregate state type with the accumulator output (#9340)
* fix: align ordered aggregate state type with the accumulator output
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: reject mismatched aggregate states and keep hard-ordered aggregates unsplit
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: keep WITHIN GROUP aggregates splittable and relabel state fields by position
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit affc0a1b1d)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): stop parallel flat scan tasks once the receiver is dropped (#9368)
* fix(mito2): stop parallel flat scan tasks once the receiver is dropped
spawn_flat_scan_task ignored send errors, so after a query was cancelled the
detached task kept reading its source to EOF. Break out of the loop when the
receiver is gone.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore(mito2): log when a parallel scan task stops on a dropped receiver
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore(mito2): include the send error in the parallel scan stop log
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 411c0f173e)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(catalog): skip backend batch_get on full cache hits (#9431)
fix(catalog): return cached batch values without an empty backend call
Count backend batch_get invocations in regression tests and cover empty input.
Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit acebd6b35b)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: coalesce equal histogram quantile bucket boundaries (#9451)
* fix: coalesce equal histogram quantile bucket boundaries
Merge numerically equivalent bucket boundaries before monotonicity repair, preserving the allocation-free path for unique boundaries. Share coalescing with histogram_fraction and cover duplicate boundaries with unit and SQL regressions.
Fixes#9443
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: keep duplicate infinity buckets in one histogram group
Confirm classic histogram completion at a series/timestamp boundary or EOF instead of the first positive infinity. Retain lookahead across batches and reject incomplete optimistic groups so equivalent infinity labels reach numeric coalescing together.
Add stream and SQL regressions for duplicate infinity labels, including established bucket counts and EOF. Correct older fixtures to use distinct keys for distinct histogram groups.
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: avoid buffering histogram groups with invalid trailing bounds
Discover candidate bucket counts at the first group boundary and let optimistic validation fall back to safe mode for malformed bounds. Document discovery and cover trailing invalid and null labels with a focused streaming regression.
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit 97913b4b3a)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(otlp): coerce empty and "-" trace attribute strings to null (#9461)
* fix(otlp): coerce empty and "-" trace attribute strings to null
Instrumentations fill numeric attributes from absent headers with an empty
string (PSR-7 getHeaderLine returns ""). Coercing that into an existing
numeric column failed and rejected the whole span.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: trim redundant row from empty-string rewrite test
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit d640ab8bc5)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(metasrv): retry failed heartbeat address updates (#9437)
* fix(metasrv): retry failed heartbeat address updates
Signed-off-by: WenyXu <wenymedia@gmail.com>
* refactor(metasrv): move shared heartbeat helpers to utils
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 65b9a35692)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: align TWCS assignment with half-open time windows (#9464)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
(cherry picked from commit 5126623e63)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(promql): keep vector labels when a computed scalar is on the right (#9478)
* fix(promql): keep vector labels when a computed scalar is on the right
`metric + time()` or `metric * scalar(other)` planned the join with the
scalar side's context, which has no tags, so every series collapsed into
one unlabeled series. Use the vector side's context when the right operand
is scalar-typed.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): update results for labels kept with a computed scalar on the right
v * pi() and v + time() now keep the vector's labels and time index name,
as in Prometheus.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): qualify columns of an unnamed operand in vector joins
A comparison filter's projection has no table name, so after the join its
time index resolved unqualified and was ambiguous when the other operand had
the same column. Also keep the empty-metric branch unchanged for scalar
left operands.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): cover comparison filters joined with a scalar
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 6da5352c53)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(promql): keep metric name for comparison operators (#9332)
* fix(promql): keep metric name for comparison operators
`collect_metric_names` dropped the response `__name__` for every binary
operator except `and` / `or` / `unless`, so comparisons such as
`node_load1 > 5` came back without the metric name. Grafana rendered them
as `{instance="..."}` and alert templates reading `$labels.__name__` got
nothing.
Follow Prometheus' `shouldDropMetricName` / `resultMetric`: arithmetic
operators drop the name, comparisons keep the left-hand name, except when
the comparison returns a bool or is a one-to-one `on(...)` match. A
`group_left` / `group_right` modifier does not reduce the labels, so the
name is kept there.
Both the HTTP PromQL API and the gRPC gateway share
`retrieve_metric_name_and_result_type`, so one fix covers both.
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* docs: document binary_keeps_metric_name
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* fix(promql): handle on(__name__), scalar lhs and group_right
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* chore(promql): correct result-side docs and drop unreachable String arm
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* fix(promql): drop metric name when explicitly ignored
Handle ignoring(__name__) for one-to-one comparisons and cover
comparison, group_left, group_right and unless behavior.
Signed-off-by: Aumnertic <hznuzc@gmail.com>
---------
Signed-off-by: Aumnertic <hznuzc@gmail.com>
(cherry picked from commit c33a6cfa56)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(query): keep filter fetch when rewriting matches term (#9521)
* test: cover matches term limit dropped by constant rewrite
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): keep filter fetch when rewriting matches term
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 483d1e49a3)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* style: format #9437 test imports for the release toolchain
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test: regenerate array_agg result for release plan rendering
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: immanuwell <pchpr.00@list.ru>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: Aumnertic <hznuzc@gmail.com>
Co-authored-by: Immanuel Tikhonov <122638311+immanuwell@users.noreply.github.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: Mohd Quamar Tyagi <104281681+Tyagiquamar@users.noreply.github.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: Aum <hznuzc@gmail.com>
* refactor: use DataFusion's UDAF implementation directly
Signed-off-by: luofucong <luofc@foxmail.com>
* remove: delete how-to guide for writing aggregate functions
Signed-off-by: luofucong <luofc@foxmail.com>
* fix ci
Signed-off-by: luofucong <luofc@foxmail.com>
* refactor: port json_encode_path to datafusion udaf
Signed-off-by: Ning Sun <sunning@greptime.com>
---------
Signed-off-by: luofucong <luofc@foxmail.com>
Signed-off-by: Ning Sun <sunning@greptime.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
* feat: add geojson function to aggregate paths
* test: add sqlness results
* test: add sqlness
* refactor: corrected to aggregation function
* chore: update comments
* fix: make linter happy again
* refactor: rename to remove `geo` from `geojson` function name
The return type is not geojson at all. It's just compatible with geojson's
coordinates part and superset's deckgl path plugin.
* feat: list/array support for postgres output
* fix: implement time zone support for postgrsql
* feat: add a geohash function that returns array
* fix: typo
* fix: lint warnings
* test: add sqlness test
* refactor: check resolution range before convert value
* fix: test result for sqlness
* feat: upgrade pgwire apis
LFCandNing Sun
Ruihang Xia
Ning Sun