* fix(servers): resolve port from IPv6 bind addresses (#9142)
Signed-off-by: immanuwell <pchpr.00@list.ru>
(cherry picked from commit 5ec679fed9)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: register region migration procedures before returning IDs (#9163)
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
(cherry picked from commit c65e8d0962)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(promql): skip empty batches in SeriesDivide stream (#9178)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 5e57b00a95)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(servers): merge duplicate timeseries across RecordBatches in Prometheus remote read (#9032)
* fix(servers): merge timeseries across RecordBatches in Prometheus remote read
Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
* fix(servers): exercise recordbatches_to_timeseries and sort samples with sort_unstable_by_key
---------
Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
(cherry picked from commit 59eb63c663)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(prometheus): honor label matchers in __name__ values query (#9134)
* fix(prometheus): honor label matchers in __name__ values query
`/api/v1/label/__name__/values?match[]={pod="abc"}` dropped every matcher
other than `__name__` and returned all metrics in the schema. No error,
just the wrong list. Grafana's metrics browser sends this request, so
picking a label value there did nothing.
Selectors that only constrain `__name__` keep answering from table
metadata. A selector constraining an ordinary label now goes to the data:
scan each metric engine physical table for distinct `__table_id` in the
time range, map the ids back to metric names, then apply the selector's
own `__name__` matchers.
Only metric engine tables are covered; other engines share no column space
to scan.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(prometheus): batch-resolve metric names by table id
Building a full table-id-to-name map meant walking every table in the
schema and holding all of them in memory, just to name the handful the
scan returned. Use `tables_by_ids` instead — one batch KV read over the
ids the scan actually produced.
The catalog walk stays, but only to find the physical tables to scan.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): read an absent label as the empty string
A matcher on a label the series does not carry only worked when the table
had no column for it at all. Where the column exists but is NULL on that
row -- the norm for logical metrics sharing a metric engine physical
table, which holds the union of their label columns -- three-valued logic
dropped the row, so `host!="host1"` and `host=""` missed every metric
without a host label.
Coalesce nullable string label columns to "" for matchers that accept the
empty string, rather than only for the OTLP temporality marker. Equality
matchers are untouched; they cannot match NULL either way.
This is the Prometheus compatibility fix#8970 deliberately kept out of
its own scope. The cost is visible in the regex sqlness plan: the
predicate becomes a CASE, so the scan loses its LastRow selector and
grows a FilterExec. Only negative and empty-accepting matchers pay it.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): don't panic on pre-epoch label value bounds
`rewrite_label_values_query` unwrapped `duration_since(UNIX_EPOCH)`, which
returns an error for an instant before the epoch. `start=1969-12-31T23:59:59Z`
parses as valid RFC3339, so the request panicked instead of answering.
Recover the sign from the error branch, and report a value beyond i64
milliseconds as an error rather than wrapping the cast.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor(prometheus): drop applicable_matchers, share the distinct scan
With the planner reading an absent label as empty, the frontend no longer
needs to pre-filter matchers per physical table. Removing that exposed a
second problem: a physical table that never took a column from a logical
table exposes no `__table_id`, and projecting it failed the whole request.
Skip those tables; the only thing that can miss is a metric with no labels.
Also pulls out the plan-build-execute-collect sequence the two label value
scans had in common.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 007836ba7a)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): cancel cache construction for incomplete scans (#9254)
* fix(mito2): cancel cache construction for incomplete scans
CacheBatchBuffer spawned the background concat task and dropped its join
handle. A scan that was cancelled or failed therefore left the task alive:
it kept compacting already queued batches, and while waiting for a range
result memory permit it held them, even though without a finish command
the result can never be put into the cache.
Keep the handle and abort it when the buffer is dropped. The handle is
cleared once the task owns the finish command, so a completed scan still
populates the cache after its stream is dropped. Abort does not preempt a
concat that is already running; it takes effect the next time the task is
polled.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(mito2): wait for the permit park before cancelling the buffer
An empty buffered_batches only proves the batches were enqueued, so the
cancellation test could abort a concat task that had never been polled.
Count acquisitions that find too few permits, a test-only signal, and drop
the buffer once the task has reached that wait. Nothing awaits between the
check and the parking, so an observed increment means the caller is about
to wait for permits the test holds for the rest of the case.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 43ce07fa69)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: merge every state row in geo_path and json_encode_path (#9339)
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit cc1be82858)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: align ordered aggregate state type with the accumulator output (#9340)
* fix: align ordered aggregate state type with the accumulator output
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: reject mismatched aggregate states and keep hard-ordered aggregates unsplit
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: keep WITHIN GROUP aggregates splittable and relabel state fields by position
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit affc0a1b1d)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(mito2): stop parallel flat scan tasks once the receiver is dropped (#9368)
* fix(mito2): stop parallel flat scan tasks once the receiver is dropped
spawn_flat_scan_task ignored send errors, so after a query was cancelled the
detached task kept reading its source to EOF. Break out of the loop when the
receiver is gone.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore(mito2): log when a parallel scan task stops on a dropped receiver
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore(mito2): include the send error in the parallel scan stop log
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 411c0f173e)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(catalog): skip backend batch_get on full cache hits (#9431)
fix(catalog): return cached batch values without an empty backend call
Count backend batch_get invocations in regression tests and cover empty input.
Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit acebd6b35b)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: coalesce equal histogram quantile bucket boundaries (#9451)
* fix: coalesce equal histogram quantile bucket boundaries
Merge numerically equivalent bucket boundaries before monotonicity repair, preserving the allocation-free path for unique boundaries. Share coalescing with histogram_fraction and cover duplicate boundaries with unit and SQL regressions.
Fixes#9443
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: keep duplicate infinity buckets in one histogram group
Confirm classic histogram completion at a series/timestamp boundary or EOF instead of the first positive infinity. Retain lookahead across batches and reject incomplete optimistic groups so equivalent infinity labels reach numeric coalescing together.
Add stream and SQL regressions for duplicate infinity labels, including established bucket counts and EOF. Correct older fixtures to use distinct keys for distinct histogram groups.
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: avoid buffering histogram groups with invalid trailing bounds
Discover candidate bucket counts at the first group boundary and let optimistic validation fall back to safe mode for malformed bounds. Document discovery and cover trailing invalid and null labels with a focused streaming regression.
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
(cherry picked from commit 97913b4b3a)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(otlp): coerce empty and "-" trace attribute strings to null (#9461)
* fix(otlp): coerce empty and "-" trace attribute strings to null
Instrumentations fill numeric attributes from absent headers with an empty
string (PSR-7 getHeaderLine returns ""). Coercing that into an existing
numeric column failed and rejected the whole span.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: trim redundant row from empty-string rewrite test
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit d640ab8bc5)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(metasrv): retry failed heartbeat address updates (#9437)
* fix(metasrv): retry failed heartbeat address updates
Signed-off-by: WenyXu <wenymedia@gmail.com>
* refactor(metasrv): move shared heartbeat helpers to utils
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
(cherry picked from commit 65b9a35692)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix: align TWCS assignment with half-open time windows (#9464)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
(cherry picked from commit 5126623e63)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(promql): keep vector labels when a computed scalar is on the right (#9478)
* fix(promql): keep vector labels when a computed scalar is on the right
`metric + time()` or `metric * scalar(other)` planned the join with the
scalar side's context, which has no tags, so every series collapsed into
one unlabeled series. Use the vector side's context when the right operand
is scalar-typed.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): update results for labels kept with a computed scalar on the right
v * pi() and v + time() now keep the vector's labels and time index name,
as in Prometheus.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(promql): qualify columns of an unnamed operand in vector joins
A comparison filter's projection has no table name, so after the join its
time index resolved unqualified and was ambiguous when the other operand had
the same column. Also keep the empty-metric branch unchanged for scalar
left operands.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(promql): cover comparison filters joined with a scalar
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
(cherry picked from commit 6da5352c53)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(promql): keep metric name for comparison operators (#9332)
* fix(promql): keep metric name for comparison operators
`collect_metric_names` dropped the response `__name__` for every binary
operator except `and` / `or` / `unless`, so comparisons such as
`node_load1 > 5` came back without the metric name. Grafana rendered them
as `{instance="..."}` and alert templates reading `$labels.__name__` got
nothing.
Follow Prometheus' `shouldDropMetricName` / `resultMetric`: arithmetic
operators drop the name, comparisons keep the left-hand name, except when
the comparison returns a bool or is a one-to-one `on(...)` match. A
`group_left` / `group_right` modifier does not reduce the labels, so the
name is kept there.
Both the HTTP PromQL API and the gRPC gateway share
`retrieve_metric_name_and_result_type`, so one fix covers both.
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* docs: document binary_keeps_metric_name
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* fix(promql): handle on(__name__), scalar lhs and group_right
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* chore(promql): correct result-side docs and drop unreachable String arm
Signed-off-by: Aumnertic <hznuzc@gmail.com>
* fix(promql): drop metric name when explicitly ignored
Handle ignoring(__name__) for one-to-one comparisons and cover
comparison, group_left, group_right and unless behavior.
Signed-off-by: Aumnertic <hznuzc@gmail.com>
---------
Signed-off-by: Aumnertic <hznuzc@gmail.com>
(cherry picked from commit c33a6cfa56)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* fix(query): keep filter fetch when rewriting matches term (#9521)
* test: cover matches term limit dropped by constant rewrite
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
* fix(query): keep filter fetch when rewriting matches term
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
---------
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
(cherry picked from commit 483d1e49a3)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* style: format #9437 test imports for the release toolchain
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
* test: regenerate array_agg result for release plan rendering
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
---------
Signed-off-by: immanuwell <pchpr.00@list.ru>
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: Tyagiquamar <mohdquamartyagi@gmail.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Signed-off-by: Aumnertic <hznuzc@gmail.com>
Co-authored-by: Immanuel Tikhonov <122638311+immanuwell@users.noreply.github.com>
Co-authored-by: jeremyhi <jiachun_feng@proton.me>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: Mohd Quamar Tyagi <104281681+Tyagiquamar@users.noreply.github.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: Aum <hznuzc@gmail.com>
* feat: add json_object scalar function
Builds a JSONB object from interleaved (key, value, ...) arguments, like
MySQL's JSON_OBJECT. Values are written into the binary directly, so
JSON-hostile characters (quotes, backslashes, control characters) need no
text-level escaping. Keys must be non-NULL strings; values may be strings,
numbers, booleans, or NULL (JSON null).
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: build entity-graph JSON objects with json_object
The derivation assembled entity_id_attrs and descriptive by concatenating a
JSON text and parsing it, escaping only backslash and double quote in runtime
values. A label containing a control character (e.g. a newline) produced
unparseable text and failed the whole semantic_entities scan instead of one
attribute. json_object assembles the JSONB binary directly from the value
columns, so no text escaping is involved; NULL-to-'' stays at the call site.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: trim comments and fold duplicate test coverage
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: json_object() returns an empty object; narrow values to integers and floats
MySQL's JSON_OBJECT allows an empty pair list, so the signature accepts zero
arguments and the row count falls back to number_rows. Decimals stay rejected
instead of casting to Float64: JSONB numbers (i64/u64/f64) cannot represent
them exactly and a silent precision loss is worse than an explicit cast.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: document key-to-string conversion and align test naming
Keys follow MySQL JSON_OBJECT: any castable type is converted to string.
Rustdoc and the cast-failure message now say so, with a numeric-key test.
Test names take the module-conventional test_ prefix.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: allow ipv4_num_to_string to accept valid integers
Signed-off-by: Johannes Sluis <joesluis51@gmail.com>
* test: update sqlness result file
Signed-off-by: Johannes Sluis <joesluis51@gmail.com>
* fix: use coercible integer signature for ipv4_num_to_string
Signed-off-by: Johannes Sluis <joesluis51@gmail.com>
---------
Signed-off-by: Johannes Sluis <joesluis51@gmail.com>
* feat: initial function rewriter for json_get
* feat: make sure rewrite rule is applied
* feat: keep analyzer's default rules
* feat: implement rewriter for arrow_cast
* test: add unit test for tht rewriter
* chore: format
* refactor: extract some more functions
* Apply suggestion from @waynexia
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
---------
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
* feat: add return_field_from_args
* feat: add JsonGetWithType
* port json_get_float and json_get_bool to new implementation, add
json_get with third argument accepting a scalar value for type.
* fix: lint fix
* chore: add sqlness tests
* chore: update tests
* feat: support function aliases and add MySQL-compatible aliases
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: get_table_function_source
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor: add function_alias mod
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: license
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(mysql): add SHOW WARNINGS support and return warnings for unsupported SET variables
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat(function): add MySQL IF() function and PostgreSQL description functions for connector compatibility
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: show tables for mysql
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: partitions table in information_schema and add starrocks external catalog compatibility
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor: async udf
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: set warnings
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: impl pg_my_temp_schema and make description functions simple
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: add test for issue 7313
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: apply suggestions
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: partition_expression and partition_description
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: test
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: unit tests
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: saerch_path only works for pg
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: improve warnings processing
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: warnings while writing affected rows and refactor
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: improve ShobjDescriptionFunction signature
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor: array_to_boolean
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat(expr): support vec_elem_avg function
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* feat: support vec_avg function
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* test: add more query test for avg aggregator
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* fix: fix the merge batch mode
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* refactor: use sum and count as state for avg function
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* refactor: refactor merge batch mode for avg function
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
* feat: add additional vector restrictions for validation
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
---------
Signed-off-by: Alan Tang <jmtangcs@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
* feat: adds format, regex_extract function and more type tests
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix: forgot functions
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* chore: forgot null type
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: forgot date type
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* feat: remove format function
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test: update results after upgrading datafusion
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* refactor: use DataFusion's UDAF implementation directly
Signed-off-by: luofucong <luofc@foxmail.com>
* remove: delete how-to guide for writing aggregate functions
Signed-off-by: luofucong <luofc@foxmail.com>
* fix ci
Signed-off-by: luofucong <luofc@foxmail.com>
* refactor: port json_encode_path to datafusion udaf
Signed-off-by: Ning Sun <sunning@greptime.com>
---------
Signed-off-by: luofucong <luofc@foxmail.com>
Signed-off-by: Ning Sun <sunning@greptime.com>
Co-authored-by: Ning Sun <sunning@greptime.com>
* feat: process id for session, query context and postgres
Signed-off-by: Ning Sun <sunning@greptime.com>
* feat: add sql functions to retrieve connection/process id
Signed-off-by: Ning Sun <sunning@greptime.com>
---------
Signed-off-by: Ning Sun <sunning@greptime.com>
* feat: add vec_kth_elem function
Signed-off-by: pikady <2652917633@qq.com>
* code format
Signed-off-by: pikady <2652917633@qq.com>
* add test sql
Signed-off-by: pikady <2652917633@qq.com>
* change indexing from 1-based to 0-based
Signed-off-by: pikady <2652917633@qq.com>
* improve code formatting and correct spelling errors
Signed-off-by: pikady <2652917633@qq.com>
* Update tests/cases/standalone/common/function/vector/vector.sql
I noticed the two lines are identical. Could you clarify the reason for the change? Thanks!
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
---------
Signed-off-by: pikady <2652917633@qq.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
Whis Liao
discord9
Joe Sluis
Ning SunandRuihang Xia
Sicong Hu
Alan TangandYingwen
codephage
Zhenchi
pikadyandZhenchi
Sicong Hu
xiaoniaoyouhuajiang