Commit Graph
6118 Commits
Author SHA1 Message Date
discord9andDennis Zhuang 528ceb7733 perf(promql): reuse sliding min and max candidates (#9099)
* perf(promql): reuse sliding min and max candidates

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): simplify extrema benchmark parameters

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): record baseline sliding extrema SQL results

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* perf(promql): rescan windows that barely overlap

Reusing candidates loses to a plain scan when consecutive windows overlap
little: the deque bookkeeping then costs more than the rescan it replaces.
A local Criterion run on 4096 samples at width 240 / step 240 measured
10.79 -> 22.14 us for min and 12.83 -> 19.85 us for max.

Pick the evaluator once per batch from the first two windows. RangeManipulate
emits one window length and one step per batch, so that sample decides for all
of them, and both evaluators return identical bits, so a wrong pick costs time
only. Batches that do not qualify fold each window on its own.

Move the incremental state into SlidingExtrema so tests can drive it directly:
the exhaustive four-sample differential test cannot reach it through a UDF
call, because such a batch never qualifies for reuse.

Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>

* fix(promql): select the extrema evaluator from batch averages

Reading the window shape off the first two windows misreads the batch.
RangeManipulate starts a series at max(query start, first aligned sample),
so a series that begins inside the query range gets a first window covering
roughly one step, and a window covering no sample at all is emitted as
(0, 0). Either one closed the gate for the whole batch, including the
one-hour window at a 15s step that candidate reuse was written for.

Compare the batch averages instead: at least 32 samples per window, and a
step advancing at most a quarter of that. Uniform batches select exactly as
before, so the thresholds keep the meaning they were measured with.

The 32-sample rule had also moved most of the benchmark and query-regression
shapes onto the rescan, including the case built to measure reset and
rebuild. Widen those windows to 40 samples, add a step at the selection
boundary, and add an end-to-end case with 40-sample windows advancing 5.

Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>

* fix(promql): ignore empty windows when measuring batch advance

The advance was read from the first and last window offsets, but a window
covering no sample is emitted as (0, 0). A query whose last evaluation lands
exactly one window past the last sample ends on such a window, and its zero
offset made a batch of disjoint windows look like one that never moved, which
selected the evaluator built for overlap. Results stayed correct; the cost was
deque bookkeeping on the shape the scan fallback exists for.

Take the offset span over the windows that cover a sample. Empty windows stay
in the window count, where they only make both conditions stricter.

Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
Co-authored-by: Dennis Zhuang <xzhuang@greptime.com>
2026-09-17 03:18:21 +00:00
Lei, HUANG 259b42c039 fix(mito2): preserve last-non-null values across compaction (#9169)
* fix(mito2): preserve last-non-null values during compaction

Use a dedicated LastNonNullPicker to close TWCS seeds over overlapping SSTs, merge connected seed groups, and defer closures with busy inputs. Keep LastRow and strict-window selection unchanged.

Index overlap candidates on the blocking compaction runtime and cover field promotion, transitive dependencies, expiration, and conservative key-range pruning with regression tests.

Refs #9146

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(mito2): guard last-non-null compaction against pending memtables

Track conservative memtable sequence lower bounds and capture them from the same version as compaction inputs. Defer unsafe SST closures and strict-window plans until pending writes no longer intersect their sequence span.

Cache minimum sequences during bulk conversion and expose a lightweight memtable getter to avoid rescanning rows or computing full statistics.

Fixes #9157

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(mito2): limit eligible last-non-null compaction outputs

Enumerate all TWCS seeds for LastNonNull while retaining bounded window planning concurrency. Apply the output budget after closure expansion and busy/memtable safety checks, preserving execution priority and ordinary TWCS behavior.

Cover rejected and absorbed seeds, priority ordering, and bounded and unlimited output counts with real-picker regression tests.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(mito2): include all candidates before last-non-null closure

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* refactor(mito2): clarify last-non-null compaction closure

Extract transitive closure expansion from output scheduling and clarify seed handling. Document the dedicated picker rationale and schema-evolution risks in PK overlap checks.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(mito2): cover deferred non-seed compaction bridges

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(mito2): defer strict-window compaction with busy dependencies

Keep busy SSTs when building the LastNonNull strict-window dependency closure, then defer the entire rewrite plan if any selected input is busy. Preserve LastRow filtering and expired-file cleanup.\n\nCover requested, transitive and unrelated busy windows, including retry after the dependency is released.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* docs(mito2): document compaction overlap index fields

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
2026-09-16 16:41:35 +00:00
Lei, HUANG d6a8974613 perf(mito2): optimize materialized column index updates (#9182)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
2026-09-16 16:28:12 +00:00
Weny Xu 4ecec69bee ci: add manual agent observability benchmarks on Aliyun ECS (#9179)
* ci: add manual agent observability benchmarks on Aliyun ECS

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: configure observability ECS budgets and reuse runner actions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: default observability runners to ecs.c9i.2xlarge

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(ci): prepare observability Docker access during ECS bootstrap

Signed-off-by: WenyXu <wenymedia@gmail.com>

* docs: remove standalone observability CI guide

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-09-16 14:47:34 +00:00
Weny Xu 809283e6b4 feat: add prepared batch write primitives (#9186)
* refactor: expose shared operator test fixtures

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: convert prepared table rows to Arrow batches

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: write prepared table batches through bulk insert

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: define prepared batch submission interface

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: validate prepared batch values and clarify write contracts

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: reuse JSON2 array schema alignment for prepared writes

Signed-off-by: WenyXu <wenymedia@gmail.com>

* ci: refresh compatibility window for v1.2.1

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-09-16 14:34:22 +00:00
discord9 3942d60042 chore(ci): update compatibility test window to v1.2.1 (#9193)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-16 13:57:45 +00:00
discord9 f5428f8a6a test: regenerate expired TLS certificates for integration fixtures (#9196)
* test: regenerate expired TLS certificates for integration fixtures

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: restore root.srl serial file for TLS fixtures

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* chore(ci): update compatibility test window to v1.2.1

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add proper TLS extensions to regenerated integration certificates

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-09-16 13:48:30 +00:00
Yingwen 9d6c2f6c58 fix(ci): rerun semantic PR checks after pushes (#9190)
Signed-off-by: evenyag <realevenyag@gmail.com>
2026-09-16 10:19:38 +00:00
Weny Xu e0915a8380 refactor: reuse common batching components in Prom ingestion (#9114)
* refactor: compose Prom ingestion with common batcher components

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: share bulk insert IPC encoding

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: centralize batcher worker channel creation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: share timestamp extraction for batched writes

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: preserve sharded batcher worker lookup

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: align batcher worker lookup and configuration validation

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: validate batcher limits inside fallible constructors

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-09-16 09:21:49 +00:00
discord9 d185a8bf76 feat: use exact sequence ranges for incremental Flow reads (#9165)
* fix: preserve structured query errors through distributed execution

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* feat: use exact sequence ranges for capable incremental flow sources

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: update SQL expectations for preserved query error codes

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* style: format exact sequence recovery tests

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: exercise dispatched exact delta failure recovery

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: record flushed exact sequence flow results

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-16 08:26:58 +00:00
Dhruv Vaishnav 7e2a75f771 feat: allow re-enabling WAL after disabling (#9130)
* feat: allow re-enabling WAL after disabling

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>

* test: regenerate skip_wal sqlness result

The previous commit changed tests/cases/standalone/common/skip_wal.sql without
regenerating the matching .result, so every Sqlness suite failed on the
mismatch. tests/cases/distributed/common is a symlink to standalone/common, so
the single stale file accounted for all five failing variants.

Regenerated from a real run. The recorded output now covers the cases the .sql
added:

* A table created with skip_wal = 'true' has no real WAL provider, so enabling
  WAL is refused with Unsupported rather than the previous InvalidArguments.
* A RaftEngine-backed table accepts the true -> false transition, and repeating
  it is a no-op.
* SHOW CREATE TABLE reports skip_wal = 'false' after the transition, including
  after a restart.
* A row written while WAL was skipped is absent after restart, while a row
  written after WAL was restored survives.

Verified locally with nothing else running on the machine: sqlness skip_wal
passes in both the standalone and distributed environments, and the regenerated
file is byte-identical to the output of an independent earlier run.

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>

* test: strengthen skip_wal provider coverage

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>

* Fix WAL provider handling during re-enable

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>

* Refactor WAL provider check

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>

---------

Signed-off-by: dhruvxvaishnav <dhruvvaishnav687@gmail.com>
2026-09-16 07:52:59 +00:00
discord9 5e57b00a95 fix(promql): skip empty batches in SeriesDivide stream (#9178)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-16 07:37:15 +00:00
discord9 8af3a04ed7 fix: preserve structured query errors through distributed execution (#9161)
* fix: preserve structured query errors through distributed execution

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: update SQL expectations for preserved query error codes

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-16 06:28:15 +00:00
Lei, HUANG 7aacc947b3 perf(mito2): lazily extract sparse primary key index values (#9176)
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
2026-09-16 06:15:52 +00:00
fys 1c01b541ec refactor(json2): JSON2 parquet projection and schema alignment (#9137)
* refactor(mito2): make JSON schema alignment targets explicit

Signed-off-by: fys <fengys1996@gmail.com>

* fix: cr

* fix: cargo fmt

---------

Signed-off-by: fys <fengys1996@gmail.com>
2026-09-16 04:44:32 +00:00
Lanqing Yang 67a403757d Perf/http sql limit materialization (#9148)
perf(servers): limit HTTP SQL rows before materialization

Signed-off-by: lyang24 <lanqingy93@gmail.com>
2026-09-16 03:46:11 +00:00
jeremyhi d7ada1761d feat: export logical tables from Metric physical scans (#9159)
* feat: add physical Metric table exporter

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: drain Metric export writes before cancellation cleanup

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* perf: construct Metric export error context lazily

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: name the logical table export entry point

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: share Parquet writer for logical table exports

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: preserve Parquet destinations and cancellation boundaries

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: clarify logical table export field names

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: clarify logical table export helper responsibilities

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: validate logical export membership by table ID

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test: simplify logical table export coverage and strengthen assertions

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2026-09-16 02:48:47 +00:00
discord9andNing Sun 94d7e2c7fc feat!: upgrade DataFusion to 55 (#8555)
* feat!: upgrade DataFusion dependencies to 55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor: migrate DataFusion 55 APIs

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: preserve table function planning behavior

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: preserve PostgreSQL query compatibility

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: preserve distributed execution plan behavior

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: cover DataFusion 55 behavior regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: update DataFusion 55 SQLness expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: complete DataFusion 55 test API migration

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: address DataFusion 55 CI regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: address remaining DataFusion 55 regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: adapt latest base code to DataFusion 55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: normalize environment-specific DataFusion 55 plans

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: align final DataFusion 55 expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: isolate DataFusion 55 regression cases

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: preserve empty result schema in timestamp widening

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: preserve JSON source column order

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* chore: use released DataFusion 55 integrations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: adapt latest execution plan mock to DataFusion 55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: pin DataFusion recursive schema and date repairs

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(promql): align dictionary temporality match keys

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: retain Greptime DataFusion fork behaviors on version 55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: restore ordinary function error expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: refresh distributed count compatibility plan

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(query): adapt last-row cast hint to DataFusion 55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: refresh instant last-row empty results for Arrow 59

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* style: simplify DataFusion expression visitor imports

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: restore sorting and PostgreSQL column-order assertions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(function): restore primitive numeric coercion signatures

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor(function): share geo integer signature types

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: cover timestamp widening overflow boundaries

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: fix decimal coercion regression imports

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(function): preserve scalar count_hash NULL state semantics

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: simplify decimal clamp case type inference

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: retain historical count_hash wrapper result

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: restore timestamp widening equality and IN pruning

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: carry upstream aggregate dynamic filter correctness fix

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: carry upstream null and predicate simplification fixes

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: restore baseline JSON ordering expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: restore histogram JSON ordering expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: refresh empty PromQL range result schemas

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: align native timestamp plan with DF55 decimal display

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: refresh native timestamp SQLness results for DF55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: regenerate NULL sample empty result headers for DF55

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: use DF55 child replacement API in timestamp regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: expose pushed scan dynamic filters to DF55 producers

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: encode string-backed PostgreSQL OID aliases in binary results

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: verify REGPROC binary and text over PostgreSQL protocol

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: register real PostgreSQL catalogs in server fixtures

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: complete DF55 expression inventories for custom query plans

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: correct RangeSelect expression fixture and column identities

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* ci: wait for Kafka WAL helper deployment rollout

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: update custom storage empty result headers

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: require exact row counts in scan statistics

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: suppress deprecated partition_statistics warning in test

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
2026-09-15 11:42:38 +00:00
jeremyhi c65e8d0962 fix: register region migration procedures before returning IDs (#9163)
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2026-09-15 10:33:35 +00:00
LFC 9f6a79da30 fix: support native JSON2 row inserts over gRPC (#9145)
* fix: support native JSON2 row inserts over gRPC

Signed-off-by: luofucong <luofc@foxmail.com>

* test: cover unknown JSON2 schema compatibility

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
2026-09-15 10:19:48 +00:00
discord9 da7e31842e perf(servers): avoid rebuilding JSON records payload (#9160)
* perf(servers): avoid rebuilding JSON records payload

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: trim json response optimization unit tests

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* perf: move bulk response items into json payload

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-15 10:02:56 +00:00
Lei, HUANG 6f10c0cd22 perf(metric-engine): avoid deep-cloning physical column metadata in verify_rows (#9162)
verify_rows deep-cloned the whole HashMap<String, ColumnMetadata> of the
physical region on every put batch. On a datanode serving wide physical
tables this showed up as ~18% of total CPU in a CPU flame graph (HashMap
clone + RawTable/ColumnMetadata drop).

Wrap physical_columns in an Arc inside PhysicalRegionState and take a
cheap Arc snapshot instead; add_physical_columns now goes through
Arc::make_mut.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
2026-09-15 09:21:53 +00:00
dennis zhuang 7cbd20a053 fix(mysql): strip leading comments before the federated statement filter (#9156)
* fix(mysql): strip leading comments before the federated statement filter

JDBC clients prefix every statement with a comment. DataGrip sends
`/* ApplicationName=DataGrip <version> */` in front of each one, and every
pattern in the federated filter is anchored with `^`, so the prefix makes all
of them miss.

Two failures follow. `SET TRANSACTION READ WRITE` reaches the SQL parser and is
rejected. Worse, the DBeaver-specific entries such as
`^(/\* ApplicationName=(.*)SELECT @@(.*))` do match DataGrip's prefix, so
`SELECT @@GLOBAL.event_scheduler` and `SHOW VARIABLES LIKE ...` are absorbed
into a zero-column output, which the MySQL writer sends as an OK packet. The
JDBC driver reports that the statement returned no cursor.

Strip leading whitespace and comments before matching, and drop the
ApplicationName-specific entries the stripping makes redundant. The three that
had no unprefixed counterpart (`SHOW PLUGINS`, `SHOW ENGINES`, `SHOW @@...`)
keep their behaviour as plain patterns.

Also covers gaps found while probing the same path:

- `BEGIN` is absorbed like `START TRANSACTION`/`COMMIT`/`ROLLBACK`.
- `SHOW [GLOBAL|SESSION|LOCAL] VARIABLES|STATUS` parses; the scope is ignored
  because GreptimeDB keeps no global/session split.
- `USER()`, `CURRENT_USER()`, `SYSTEM_USER()` and `SCHEMA()` are registered.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(mysql): keep executable comments, reject multi-statement absorption

Review follow-up on the comment stripping, plus the remaining MySQL
compatibility gaps from #9155.

`/*!...*/` is an executable comment: mysqldump emits its initialization as
`/*!40101 SET NAMES ... */`, and the patterns match those verbatim. Stripping
it left an empty statement, so nothing matched and the original SQL reached the
parser, which rejects `SetNames` and `MultipleAssignments`. Leave executable
comments in place.

Absorbing a request also has to stop at a statement boundary, because every
pattern ends in `(.*)`. `BEGIN; INSERT INTO t VALUES (1)` used to fail on the
unsupported `BEGIN`; once `BEGIN` became absorbable the whole request would
report success and write nothing. A request is now scanned for a second
statement and handed to the query engine if it has one. The scan skips plain
comments and string literals so a `;` inside either is not a boundary, and
treats a `/*!...*/` that is not the request itself as a statement, since it
carries SQL.

`check()` now dispatches on the leading statement keyword, so INSERT, UPDATE,
CREATE and ordinary SELECTs run no regex at all — this replaces the narrower
hand-rolled INSERT shortcut. The statement scan runs only for a request the
patterns already matched, which is always a short one.

New compatibility surface:

- `SELECT CURRENT_USER|SESSION_USER|SYSTEM_USER|USER` without parentheses, and
  `SELECT @var`, are answered here rather than in the parser. Both are anchored
  to the whole statement, so `SELECT user FROM t` still reads the column.
- `information_schema.plugins`, `user_privileges` and `processlist` are
  registered as empty tables, like the other MySQL-shape tables around them.
  Sessions are reported through `information_schema.process_list` and
  `SHOW PROCESSLIST`; `processlist` carries the column shape only.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
2026-09-15 08:59:30 +00:00
Yingwen 7c85798ad0 feat(mito2): reconcile series indexes in background (#9086)
* feat(mito2): reconcile series indexes in background

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): clean up series indexes published during region drop

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(config): align series index examples with upstream enable flag

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): run series index tasks on compaction runtime

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2026-09-15 08:44:18 +00:00
Weny Xu d9b97796a6 fix: preserve count correctness after repartition (#9154)
* fix: preserve count correctness after repartition

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: restore full predicate guard for count statistics

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: update count plans for full predicate guard

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: preserve count statistics for safe partition scans

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: retain repartition home guard and cover staging flush

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-09-15 08:26:57 +00:00
Lei, HUANG e0552eee2d fix(mito2): preserve effective sequences during compaction (#9147)
* fix(mito2): preserve effective row sequences during compaction

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(mito2): inherit compaction input sequence bounds

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
2026-09-15 07:43:16 +00:00
jeremyhi 32bf865ffb docs: propose Metric export and import optimizations (#9121)
* docs: propose Metric export and import optimizations

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* docs: colocate Metric RFC supporting materials

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* docs: condense Metric PoC evidence into readable results

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* docs: use the tracking issue as the implementation plan

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* docs: clarify Metric export design and ordering acceptance

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2026-09-15 07:18:38 +00:00
Immanuel Tikhonov 5ec679fed9 fix(servers): resolve port from IPv6 bind addresses (#9142)
Signed-off-by: immanuwell <pchpr.00@list.ru>
2026-09-15 06:32:38 +00:00
discord9 ba0f7acd93 feat(mito2): add opt-in byte-stream-split encoding for float SST fields (#9069)
* feat(mito2): add opt-in byte stream split encoding

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(mito2): correct float encoding checks

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(compat): cover float SST encoding

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(mito2): compile float encoding tests

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(mito2): release parquet test writer

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(mito2): register float test primary key

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(mito2): verify BSS write lifecycles

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(metric-engine): verify BSS physical SST

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(mito2): verify bulk BSS lifecycle

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(mito2): compile bulk BSS test

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor(mito2): narrow bulk encoding constructors

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(compat): accept generated float upgrade output

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(compat): accept generated float downgrade output

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor(mito2): narrow bulk encoding builder

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(perf): add default versus BSS storage comparison

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(perf): align BSS reader benchmarks with prior study

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(perf): parse current read benchmark averages

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(perf): retain default float encoding in direct SST fixtures

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(perf): isolate BSS user SSTs and benchmark every file

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* docs(perf): record measured BSS storage and reader tradeoffs

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* docs(perf): expose warm scan variability and evidence limits

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* docs(perf): clarify BSS baseline and storage measurement scope

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(perf): model bounded mixed integer and fractional metric series

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* docs(perf): report bounded mixed BSS measurements and query regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* docs(perf): qualify timings affected by concurrent host builds

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(perf): include float BSS comparison in default regression cases

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(perf): omit unsupported float encoding option from baseline setup

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-14 12:05:42 +00:00
discord9 03d4e62a22 fix(mito2): deliver complete WAL entries without waiting for next input (#8916)
* fix(mito2): deliver complete WAL entries without waiting for next input

Entry completeness is self-contained via Entry::is_complete(); no
lookahead to the next entry is needed. Decode and yield complete entries
immediately in both the log-store reader and the entry distributor.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(mito2): simplify live WAL entry fixtures

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-14 11:16:22 +00:00
Weny Xu 737025760e feat: support request-level WAL skipping for bulk inserts (#9110)
* feat: support request-level WAL skipping for bulk inserts

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: cover bulk insert WAL skipping across protocols

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: align WAL snapshot naming with sequence watermarks

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: bump proto

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-09-14 09:38:05 +00:00
Yingwen 23a1a5ecaa refactor: remove constant vector and replicate operation (#8999)
* refactor: remove constant vector

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: remove vector replicate operation

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: preserve scalar vector types with optional type hints

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: remove obsolete mutable vector helper and import

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: preserve typed nulls in struct scalar conversion

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2026-09-14 09:25:33 +00:00
Yingwen a23fe1c2dc fix: configure series indexes with an enable flag (#9141)
* fix: use join_dir for series index config path

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: configure series indexes with an enable flag

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: omit experimental series index from example configs

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: preserve legacy cache cleanup path behavior

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: remove trivial path joining tests

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: isolate worker group WAL directories on Windows

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2026-09-14 09:21:31 +00:00
dennis zhuang 577de012b2 perf(servers): group Prometheus response rows by label runs (#9092)
* perf(servers): group Prometheus response rows by label runs

Query output tends to be clustered by series, but a matrix response read
the same label values out of the tag columns once per row, and allocated
a key vector per row to look the series up.

Use `arrow::compute::partition` to find the runs of rows that share their
labels and build the series key once per run. The key buffer is hoisted
out of the row loop and handed to the map only when the series is new,
through the raw entry API so the key is hashed once either way.

Partitioning does not pay off when rows are not clustered, so a few
adjacent row pairs are probed first to pick between the run path and the
row-by-row path. Both paths produce the same series.

Drop the per-row "same labels as the previous row" check from #8815. Runs
cover the clustered case it was written for, and it now costs more than
it saves: 10% on a result with one row per series, 1-3% on clustered ones.

Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>

* refactor(servers): find label runs with cmp::distinct

`arrow::compute::partition` computes the same ranges on the same kernel,
but its contract takes lexicographically sorted columns, and query output
is not sorted: range queries run without the plan's output sort since
#9090, `sort`/`topk` order by value, and the tag column order in the
schema does not have to match any sort key. An implementation that
exploited the precondition would merge `a, b, a` into one run and
attribute one series' samples to another, without failing.

`cmp::distinct` is element-wise, so it holds for any row order, and its
null handling is the one a series key needs: a null label and an empty
one are distinct, two nulls are not. Building the ranges from the
boundary bitmask also folds away the `tag_columns.is_empty()` case, since
no columns means no boundaries means a single run.

Same kernel, so the benchmark does not move: -1.4% to +1.4% across shapes
with no consistent sign, against +-3% run-to-run drift.

Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>

---------

Signed-off-by: Dennis Zhuang <xzhuang@greptime.com>
2026-09-14 09:01:09 +00:00
jeremyhi 0059508e29 perf: avoid directory listing for explicit COPY input files (#9126)
* perf: avoid directory listing for explicit COPY input files

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: preserve COPY input symlink filtering

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2026-09-14 07:52:25 +00:00
discord9 13c69cdaee perf(gc): pack file reference exchange (#9009)
* perf(gc): pack file reference exchange

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(gc): address packed reference review feedback

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(gc): stop without retry when maintenance is enabled

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(meta): avoid logging malformed mailbox payloads

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-14 07:30:39 +00:00
dennis zhuang 8e45809b6d perf(query): check Substrait encodability once per plan (#9095)
`PlanRewriter::should_expand` encoded every node it ascended past to
Substrait. Each call clones the plan, runs an analyzer over it and builds
a fresh `SessionState` with all default features, so a deep PromQL plan
pays that cost once per level.

Encode the root once in `PlanRewriter::new` and skip the per-node call
when it succeeds. `to_substrait_plan` recurses from the root to the
leaves, and `should_expand` only receives sub-trees that `f_down` pushed
on the stack untouched, so a root that encodes proves the whole descent
encodes. A root that does not encode falls back to the per-node check,
which is what locates the node that has to stay on the frontend.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
2026-09-14 06:49:20 +00:00
dennis zhuang 7152ca9264 fix(promql): skip NULL samples and fix counter extrapolation order (#9118)
* fix(promql): treat NULL field values as absent samples in range functions

Range functions read the value column through `Float64Array::values()`,
which returns the raw buffer and ignores the null bitmap. A NULL field
value means the series has no sample at that timestamp, so the padding
under a null slot (0.0 in practice) was counted as a real sample.

`rate`, `increase`, `delta`, `changes`, `resets`, `idelta`, `irate`,
`quantile_over_time` and `avg_over_time` now work on samples instead of
slots. `stddev_over_time` and `stdvar_over_time` used to panic on a NULL
slot, and tokio swallowed the panic so the query returned success with
that series missing.

Null handling is gated on `null_count() == 0` over the whole backing
array, checked once per batch, so tables without NULLs keep the existing
code path. `deriv` and `predict_linear` already had this guard through
`linear_regression_slices` and are untouched.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): keep other fields when one has no sample in the window

Review follow-up. Two things surfaced once range functions started
returning NULL for a window without samples.

The filter after a function call required every field column to be
non-NULL, so on a multi-field table one field with no samples in a
window would drop the other fields' results with it. It now keeps a row
when any field has a sample, which is the shape a selector already
emits. On a single field column the two predicates are identical.

`quantile_over_time` returned NaN rather than NULL for a window without
samples, so the row survived that filter. Prometheus returns an empty
vector there, so the emptiness check now sits in the range UDF; the
shared quantile kernel still yields NaN for an empty slice, matching
upstream's `quantile()` helper that `quantile_aggr` depends on.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* docs(promql): correct two comments on the null handling

The planner one did not say why `preserve_any_value` is hardcoded at
that call site, which is the question a reader arrives with. The
`quantile_over_time` one described the empty-window behaviour while
sitting on the `has_nulls` line, and that behaviour had moved into
`window_quantile`.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix(promql): clamp extrapolation before snapping a counter to zero

Prometheus clamps `durationToStart` to half an average interval once the
first sample is past the extrapolation threshold, and only then lets the
counter zero-snap shorten it further, so the snap can never lengthen the
leading extrapolation. Running the snap first let it rescue a duration
the clamp should have cut, and `rate` and `increase` over-extrapolated
to the left. For samples 1@0s and 2@1s in a 4s window at 1s, upstream
gives 0.375 and this returned 0.5.

`extrapolation_matches_prometheus_on_seeded_windows` carries a
line-by-line port of upstream `extrapolatedRate` as an oracle and diffs
it against the UDF over seeded windows, so the order stays pinned.
`factor` also picked up upstream's guard against a zero sampled
interval, which previously divided by zero.

Two sqlness results move, both verified against the upstream algorithm.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* test(promql): fold range_presence_null into null_samples

#9104 landed its own NULL-sample case whose data is the same series this
one already used: one host with interior NULLs, one host with nothing
but NULLs. Keeping both means two files asserting the same semantics on
the same rows.

The merged case keeps every query from both, so the presence functions
still cover the trailing-NULL window, the count of two, the empty
left-open window at t=7, and the all-NULL windows.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
2026-09-14 06:33:36 +00:00
discord9 7f949f48c0 fix(promql): preserve native timestamps through sample selection (#9070)
* fix(promql): preserve native timestamps through sample selection

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(promql): retain column indices in instant plan ordering

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): update native precision plan expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(promql): preserve selector output column order

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): verify preserved selector output order

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): refresh native timestamp explain expectations

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: apply PromQL offsets without native timestamp overflow

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: cover negative PromQL offsets at native timestamp bounds

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* feat: allow native precision instant LastRow selection

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor: discard unused bounds for empty range intersections

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix: preserve native time bounds independently for LastRow

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test: verify native LastRow predicates and overflow through SQL

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* docs: explain native PromQL selection and scan invariants

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(promql): address timestamp helper and stream review feedback

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* refactor(promql): pass selector offsets explicitly from planner

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): retain explicit offset in payload overflow regression

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(promql): record inner-offset subquery SQL results

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-14 03:47:57 +00:00
Lei, HUANG 743261f05e fix(mito2): prevent JSON2 SWCS data loss from misaligned Parquet statistics due to projection (#9129)
* fix(mito2): look up row group stats by parquet leaf index for nested columns

On flat-format tables a logical column can expand to multiple parquet
leaf columns (e.g. a JSON2 struct stores the remainder and one leaf per
promoted path). ParquetFlat used the logical column index in the SST
schema directly as the leaf index when reading row group statistics, so
min/max/null stats of every column after a nested column were read from
wrong leaves.

When the misplaced leaf held order-compatible statistics (e.g. a small
Int64 JSON path vs. the timestamp window predicate), min-max pruning
dropped whole row groups by mistake. SWCS compaction reads inputs with a
time window predicate, so it silently lost all rows of such files; plain
queries with time-range predicates were affected as well.

Map each column to its first parquet leaf column and report NoStats for
columns with multiple leaves, which makes pruning conservative for them.
Add a unit test and a sqlness regression case that reproduces the data
loss on the unfixed binary.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(mito2): skip nested root stats and correct SWCS regression baseline

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(mito2): resolve statistics leaves for primary-key SST readers

Resolve scalar roots against the actual Parquet schema in shared statistics helpers, covering both flat and primary-key readers. Remove flat-side translation to avoid mapping twice and align encoded primary-key statistics as well. Cover dense flat, legacy dense and sparse layouts with statistics and time-pruning regressions.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(mito2): distinguish known null counts from unknown statistics

Assert validity before reading timestamp null counts and add a nullable scalar after the nested root with a known nonzero count. Exercise the assertions for flat and primary-key SST layouts.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(mito2): check JSON2 time pruning before SWCS compaction

Query the first time window immediately after FLUSH to cover predicate reads on flush-written SSTs independently of compaction outputs. Regenerate the sqlness expectation and retain the post-compaction checks.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(mito2): validate all business columns after JSON2 SWCS

Expand the final regression query to all eight business columns so the generated expectation verifies complete rows, including tags and scalar fields, after repeated compaction.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
2026-09-14 03:44:51 +00:00
Ning Sun ece78a4dad ci(backport): assign backport-failure issue to the original PR author (#9122)
Signed-off-by: Ning Sun <sunning@greptime.com>
2026-09-14 03:41:10 +00:00
Lei, HUANG 9696882217 fix(mito): preserve mixed JSON2 types during compaction (#9135)
* fix(mito): preserve mixed JSON2 types during compaction

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(mito): assert all restored JSON2 rows in unordered merge test

The regression test for aligning JSON2 layouts across unordered bulk
parts only round-tripped the first part's `a` values. A merge that
dropped or corrupted the second source's `b` values would still pass.

Assert the merged batch has four rows and that the restored values of
the second part (`{"b": 3}`, `{"b": 4}`) survive the merge, so the
test covers both opaque sources as its comment claims.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
2026-09-14 03:38:53 +00:00
discord9 1fce0eb6d8 fix(ci): increase query regression ECS disk to 80 GiB (#9123)
Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-14 02:45:10 +00:00
dennis zhuang 3d1278b39b docs: restructure README and add a cross-signal SQL example (#9136)
The README claimed signals could be correlated in SQL but contained no SQL
at all, and 68 lines of badges, social buttons and a hand-written TOC pushed
the first section off the opening screen.

- Add a cross-signal join over opentelemetry_traces and opentelemetry_logs
- Drop the hand-written TOC in favour of GitHub's built-in one, and collapse
  the badge block; move CI and coverage badges to Contributing
- Fold the Overview section into What's Supported, keeping the diagram
- Give Benchmarks its own section instead of nesting it under Compatibility
  and Migration, and add Agent RCA Bench
- Split the star request and user logos out of Project Status
- Replace the edge-device bullet, which nothing else in the README supports,
  with Loki query performance

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
2026-09-14 02:11:59 +00:00
discord9 555485c40e perf(promql): avoid per-window allocations in simple range functions (#9104)
* perf(promql): avoid per-window allocations in simple range functions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* perf(promql): avoid copying smoothing window values

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(perf): cover smoothing copy removal across window layouts

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* fix(promql): skip null samples in simple range functions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-11 11:59:45 +00:00
Lei, HUANG 21aed0371f feat(pprof): switch CPU profiler to framehop unwinder (#9125)
* feat(pprof): switch CPU profiler to framehop unwinder

Replace the default libgcc-based unwinder in pprof-rs with the
framehop unwinder, which is designed to be async-signal-safe:

- framehop performs no heap allocation during unwinding
  (MustNotAllocateDuringUnwind)
- It handles prologue/epilogue interruption correctly
- It falls back to frame-pointer unwinding when CFI is unavailable
- It does not depend on libgcc's unwind implementation, which is
  documented as not signal-safe (see tikv/pprof-rs#36)

Bump pprof from 0.14 to 0.15 in all three consumers (common-pprof,
cmd, servers) to unify on a single version. pprof 0.15 also replaces
parking_lot with spin-rs to avoid a potential profiler deadlock (#268).

This addresses the libgcc_s.so.1 #GP crash observed in production
when CPU profiling is active, by eliminating the libgcc unwinder
from the signal handler path entirely.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(pprof): gate framehop-unwinder to supported targets

framehop-unwinder is only available on x86_64/aarch64 Linux/macOS.
Enabling it unconditionally for all Unix targets breaks the build on
riscv64 and other platforms: pprof disables its backtrace-rs fallback
when framehop-unwinder is set, but the framehop module is not compiled
on unsupported targets, leaving no TraceImpl implementation.

Split the pprof dependency: the base target.'cfg(unix)' block carries
the common features (flamegraph, prost-codec, protobuf), and a separate
target block adds framehop-unwinder only on supported targets.
Cargo unions features from both blocks on matching targets, so x86_64
and aarch64 Linux/macOS get the full feature set while other Unix
targets fall back to the default backtrace-rs implementation.

Addresses review comment discussion_r3987313632.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
2026-09-11 09:39:13 +00:00
discord9 97648525cf perf(mito2): skip proven all-match prefilters (#9066)
* perf(mito2): skip proven all-match prefilters

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(perf): add manual all-match prefilter reproduction

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(mito2): cover all-match prefilter execution paths

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(mito2): match prefilter fixture to sparse SST schema

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* chore(mito2): address all-match prefilter lint findings

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(perf): cover all-match prefilters in default regressions

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-09-11 09:35:57 +00:00
Yingwen b8e0f8e62e feat(mito2): add series index planning and builders (#9085)
* feat(mito2): add series index planning and builders

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): track window coverage and separate index builds

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): normalize index inputs and bound SST window expansion

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): simplify series index source summaries

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore(mito2): assign series index deduplication TODO

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): initialize skip_wal in series index tests

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2026-09-11 08:11:36 +00:00
Weny Xu a673e084b2 test: cover request-level insert WAL skipping end to end (#9093)
* test: cover request-level WAL skipping end to end

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: cover session WAL policy and COPY recovery in sqlness

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix(test): isolate Mito test feature in dev dependencies

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: parameterize WAL protocol cases and make setup explicit

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: cover skip-WAL hints across streaming messages

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-09-11 07:48:57 +00:00
Lei, HUANG 9619085310 chore: bump tikv-jemalloc-sys patch to jemalloc dev (ff80bf2d) (#9119)
* chore: bump tikv-jemalloc-sys patch to jemalloc dev (ff80bf2d)

Update the [patch.crates-io] rev for tikv-jemalloc-sys from
e1846d8c (5.3.1 + 54f22c83 backport) to ff444d4 (upstream dev HEAD,
161 commits ahead of 5.3.1).

The dev branch includes additional TSD/tcache fixes beyond the
original backport:
- fb5499aa9c: Handle jemalloc calls after TSD teardown
- 61dc1da395: Fix possible tcache corruption on fiber migration
- 1e92317014: Fix thread-exit TSD cleanup

See GreptimeTeam/jemallocator branch bump-jemalloc-dev and
tikv/jemallocator#182.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* chore: point tikv-jemalloc-sys patch at GreptimeTeam/jemallocator main

GreptimeTeam/jemallocator#2 has been merged; reference the merge
commit e254a7ea on main instead of the PR head branch. Jemalloc
source content is unchanged (still upstream dev ff80bf2d).

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
2026-09-11 07:29:43 +00:00