rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-08-18 20:18:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,760 Commits 164 Branches 267 Tags
json2-flush-experiment
Commit Graph

963 Commits

Author SHA1 Message Date
Yingwen 9b579c7618 feat: make parquet row group size configurable (#8446) 
* feat: make parquet row group size configurable via max_row_group_row_count region option

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: split inverted index segments across row group boundaries

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: verify max_row_group_row_count region option

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: update row group size sqlness result

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): use parquet row group size for index rebuild

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: update query perf fixture indexer builder

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): clamp index selection to parquet rows

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: support altering parquet row group size

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): handle idempotent append mode in mixed alters

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): stage mixed region option alters atomically

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito2): ignore inverted indexes with mismatched row counts

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2026-07-23 03:56:10 +00:00
LFC eddedff769 fix(json2): treat empty object as null when insert (#8602) 
* fix(json2): treat empty object as null when insert

Signed-off-by: luofucong <luofc@foxmail.com>

* fix ci

Signed-off-by: luofucong <luofc@foxmail.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
 2026-07-22 13:09:02 +00:00
discord9 3155d49f84 fix: convert literals in joins and subqueries (#8501) 
* fix: convert literals in joins and subqueries

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve semi joins with duplicate fields

Signed-off-by: discord9 <discord9@163.com>

* test: update limit error after subquery conversion

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-22 10:48:26 +00:00
fys 4d61471cc8 feat(json2): support JSON2 nested path fallback reads (#8540) 
* feat: add nested read strategy for variant parent fallback

Signed-off-by: fys <fengys1996@gmail.com>

* refactor: drop nested fallback metadata from projection plan

Signed-off-by: fys <fengys1996@gmail.com>

* refactor: simplify nested parquet fallback planning

Signed-off-by: fys <fengys1996@gmail.com>

* add docs

Signed-off-by: fys <fengys1996@gmail.com>

* add Json2FallbackPlan

Signed-off-by: fys <fengys1996@gmail.com>

* add Json2FallbackDecoder

Signed-off-by: fys <fengys1996@gmail.com>

* test: add json2 nested fallback sqlness coverage

Signed-off-by: fys <fengys1996@gmail.com>

* add more sqlness test about json2 nested read

Signed-off-by: fys <fengys1996@gmail.com>

* refactor: split parquet reader nested stream helpers

Signed-off-by: fys <fengys1996@gmail.com>

* cache somethins in hot path

Signed-off-by: fys <fengys1996@gmail.com>

* chore: refine code

Signed-off-by: fys <fengys1996@gmail.com>

* minor change

Signed-off-by: fys <fengys1996@gmail.com>

* feat(mito2): align nested JSON2 parquet schemas

Signed-off-by: fys <fengys1996@gmail.com>

* refactor(mito2): default nested JSON reads to fallback

Signed-off-by: fys <fengys1996@gmail.com>

* chore: revert some unnecessary changes

Signed-off-by: fys <fengys1996@gmail.com>

* disallow JSON2 root fallback in parquet projection

Signed-off-by: fys <fengys1996@gmail.com>

* document JSON2 fallback root guard

* fix(json2): support variant-to-struct schema alignment

* refactor: simplify nested JSON schema alignment

* fix: cr

* fix: cr

* chore: add comment for null_on_json_type_mismatch

---------

Signed-off-by: fys <fengys1996@gmail.com>
 2026-07-22 09:42:26 +00:00
Lei, HUANG d8f9bb4e3d test: stabilize ttl instant sqlness case (#8595) 
Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
 2026-07-22 07:21:59 +00:00
discord9 8843dc9104 fix: stabilize remote analyze stage ordering (#8584) 
* fix: stabilize remote analyze stage ordering

Signed-off-by: discord9 <discord9@163.com>

* Update src/query/src/dist_plan/merge_scan.rs

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-07-22 15:57:19 +08:00
discord9 4a0dd8f9bc fix(query): preserve bare plan names in analyze json (#8519) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-22 03:33:19 +00:00
discord9 56addd0623 fix: stream remote analyze metrics while pending (#8405) 
* fix: stream remote analyze metrics while pending

Signed-off-by: discord9 <discord9@163.com>

* test: verify flight metrics preserve pending batch

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve direct SST perf queries in plans

Signed-off-by: discord9 <discord9@163.com>

* fix: bind flight metrics capability to query

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-21 04:17:50 +00:00
Lei, HUANG 6a53bb053c feat: enable soft-drop table lifecycle (#8554) 
* feat: add purge_table admin function

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: restrict purge_table to admin

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: keep purge tombstone lookup consistent

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test: verify soft-drop table lifecycle

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* ci: run soft-drop lifecycle sqlness

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: read purge tombstones authoritatively

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): invalidate soft-drop marker caches

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(meta): enable configurable table soft drop

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(ci): configure gc for soft-drop sqlness

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): close regions before soft-drop metadata

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): avoid rollback after soft-drop tombstone

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(function): validate admin single-row calls

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): log purge dropped table target

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): mark soft-drop config experimental

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: sqlness test base

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
 2026-07-20 08:28:35 +00:00
Yingwen b1580c98f6 fix: display codes in metasrv client errors (#8558) 
* fix: display codes in metasrv client errors

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: normalize meta client errors in sqlness

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2026-07-20 06:48:49 +00:00
Ruihang Xia 0479c05092 perf: preserve dictionary-encoded query labels (#8541) 
* perf: preserve dictionary-encoded query labels

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(client): skip dictionary Flight batches

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(servers): decode dictionary labels in HTTP output

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* test(mito2): support dictionary tags in series scans

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(servers): preserve dictionary child nulls in SQL

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(promql): compare dictionary tags by logical nulls

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): keep dictionary tags within query paths

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refactor(query): scope PK dictionary encoding to reads

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): preserve pushdown for dictionary labels

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): handle dictionary labels in query operators

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): complete label type matching

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): align dictionary query schemas

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): decode incompatible OR labels

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix(query): preserve dictionary partition pruning

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: handle dictionary query edge cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: import dictionary downcast macro

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-07-17 13:57:10 +00:00
Lei, HUANG bbf989ac36 feat: support soft-drop recycle bin and UNDROP TABLE (#8546) 
* feat: support full WAL retirement

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: complete close request migration

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* refactor: guard Kafka provider setup behind index collector check

Move Kafka provider initialization and `get_or_insert` inside the
existing `if let Some(collector)` block so these operations are
skipped when no global index collector is configured.

Affected file:
- `src/log-store/src/kafka/log_store.rs`

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* chore: avoid to_vec

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat: clean up soft-dropped regions offline

Use an explicit RegionCleanUp request for purge-table cleanup so tombstoned regions can be removed without reopening them.

Route cleanup through datanode, Mito, and metric-engine offline paths, including WAL obsoletion and region directory removal.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): reject file-engine soft drop

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* chore: preserve soft-drop cleanup split state

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* refactor: remove obsolete CleanUp match arm from RegionRequest

The `CleanUp` variant in the `region_request::Body` match is now handled
exclusively by `RegionServer` via a separate path. This arm would have
returned an unexpected error, so removing it eliminates dead code.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): clean every soft-dropped region replica

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): order soft-drop replica cleanup

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* Revert "fix(meta): order soft-drop replica cleanup"

This reverts commit e77162d3e5ebcf2817e2845a6a5177c328fb2c60.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* Revert "fix(meta): clean every soft-dropped region replica"

This reverts commit 2378e00cc258ca1b6a85a1aafbd68c79c666f43c.

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(catalog): expose soft drops in recycle bin

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(sql): add UNDROP TABLE

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(sql): cover UNDROP TABLE execution

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(sql): keep successful UNDROP result

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: reject stale undrop by name-based lookup after tombstone consumed

When a table is dropped, recreated under the same name (consuming the name
tombstone), and then the recreated table is dropped, an undrop procedure
that was built before the first drop and holds a stale original table name
should fail with TableNotFound instead of silently matching a different
table.

Changed `UndropTableProcedure::on_prepare` to perform a name-based lookup
when `table_name` is available and filter by table ID, ensuring that a
dropped table can only be recovered when its name tombstone still maps to
the expected ID.

- `src/common/meta/src/ddl/undrop_table.rs`: name-first lookup in on_prepare
- `src/common/meta/src/ddl/tests/drop_table.rs`: test for the stale-id
  rejection case

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(log-store): keep Kafka obsolete_all as no-op

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(catalog): hide purging tables from recycle bin

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(catalog): scope recycle bin scans by catalog

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test(sqlness): update recycle bin expectations

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* refactor: avoid redundant recycle bin allocations

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
 2026-07-17 08:15:21 +00:00
Div 1acdec9334 fix: timestamp display precision should respect column schema (#8227) (#8238) 
* fix: timestamp display precision should respect column schema (#8227)

Previously, the MySQL writer converted timestamps to NaiveDateTime via
to_chrono_datetime_with_timezone(), then passed the NaiveDateTime to
write_col(). NaiveDateTime::Display uses a fixed 6-digit fractional-
second format, causing:
  - TIMESTAMP(3) to show '.195000' instead of '.195'
  - TIMESTAMP(9) to show '.195123' instead of '.195123456'

Fix: use Timestamp::to_timezone_aware_string() directly, which formats
with chrono's '%.f' specifier — it strips trailing zeros and preserves
full nanosecond fidelity based on the actual stored unit.

Also add comprehensive unit tests in mysql_writer_test.rs covering:
  - All concrete data type → MySQL column type mappings
  - UNSIGNED_FLAG propagation for unsigned integer types
  - Timestamp precision for all four units (0/3/6/9 decimal places)
  - Edge cases: zero subseconds, trailing-zero stripping, Unix epoch,
    negative timestamps, and timezone offset shifts
  - Column-def ordering, empty schema, and decimal variants

Signed-off-by: Divyansh <anshmcs@gmail.com>

* perf: reuse format buffer for timestamp serialization to avoid per-row heap allocation

Signed-off-by: Divyansh <anshmcs@gmail.com>

* refactor: rename test variable and update visibility of create_mysql_column function

Signed-off-by: Divyansh <anshmcs@gmail.com>

* test: fix timestamp test constants and add sqlness case for display precision

- Correct the base epoch constant: 1_748_836_200 is 2025-06-02, not
  2026-06-02; use 1_780_372_200 to match the documented instant.
- Fix subsecond expectations: chrono's %.f renders fractional digits in
  groups of 3 (.100 / .010), it does not strip to .1 / .01.
- Add sqlness case timestamp_precision_display reproducing issue #8227
  over the MySQL protocol (TIMESTAMP(0/3/6/9) rendering).
- cargo fmt reflow of create_mysql_column signature.

Signed-off-by: Divyansh <anshmcs@gmail.com>

* fix: support binary protocol for timestamp column

Signed-off-by: Divyansh <anshmcs@gmail.com>

* test: update sqlness results for new timestamp display precision

Signed-off-by: Divyansh <anshmcs@gmail.com>

---------

Signed-off-by: Divyansh <anshmcs@gmail.com>
 2026-07-17 07:22:17 +00:00
discord9 a598690bf7 fix(promql): handle missing labels in or matching (#8504) 
* fix(promql): handle missing labels in or matching

Signed-off-by: discord9 <discord9@163.com>

* test(promql): streamline or matching coverage

Signed-off-by: discord9 <discord9@163.com>

* fix(promql): preserve unmatched rhs series in or

Signed-off-by: discord9 <discord9@163.com>

* perf(promql): stream union distinct inputs

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-16 02:29:43 +00:00
Yingwen 0a002f4d65 feat: support per-region write buffer limits (#8473) 
* feat(mito): add per-region write buffer limit

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat(mito): add default region write buffer size

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: expand configuration change checklist

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: cover table write buffer size option

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito): refine region write buffer checks

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito): keep region-stalled writes queued

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito): allow zero region write buffer size

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito): validate region state before write stall

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix(mito): reject writes beyond region hard limit

Signed-off-by: evenyag <realevenyag@gmail.com>

* docs: update example

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2026-07-15 07:31:25 +00:00
discord9 6e8bd8a5f9 test: add legacy JSON2 non-append compat case (#8513) 
* test: add legacy json2 non-append compat case

Signed-off-by: discord9 <discord9@163.com>

* Apply suggestions from code review

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

---------

Signed-off-by: discord9 <discord9@163.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-07-15 03:01:35 +00:00
discord9 2123108db0 fix(promql): preserve ordinary NaN samples (#8494) 
* fix(promql): distinguish stale markers from NaN

Signed-off-by: discord9 <discord9@163.com>

* fix(promql): preserve ordinary NaN samples

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-15 01:54:49 +00:00
discord9 b30765e5e9 fix: preserve distributed topk merge ordering (#8432) 
* fix: preserve distributed topk for scalar latest

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve scalar latest ordering across merge scan

Signed-off-by: discord9 <discord9@163.com>

* fix: carry merge scan ordering from planner rewrite

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan ordering metadata

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan partition ordering gate

Signed-off-by: discord9 <discord9@163.com>

* fix: require adjacent merge sort for ordering metadata

Signed-off-by: discord9 <discord9@163.com>

* test: update ordering sqlness plans

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve distributed topk merge ordering

Signed-off-by: discord9 <discord9@163.com>

* test: cover latest per series queries

Signed-off-by: discord9 <discord9@163.com>

* test: update distributed merge sort sqlness

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan ordering partition gate

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge scan over-partition ordering

Signed-off-by: discord9 <discord9@163.com>

* test: cover distributed latest with low parallelism

Signed-off-by: discord9 <discord9@163.com>

* fix: keep distributed merge sort opaque

Signed-off-by: discord9 <discord9@163.com>

* test: assert distributed scalar latest merge sort

Signed-off-by: discord9 <discord9@163.com>

* test: clarify merge scan ordering helper

Signed-off-by: discord9 <discord9@163.com>

* fix: complete merge sort exec delegation

Signed-off-by: discord9 <discord9@163.com>

* docs: explain merge sort limit pushdown

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge sort optimizer opacity

Signed-off-by: discord9 <discord9@163.com>

* docs: explain merge sort optimizer hooks

Signed-off-by: discord9 <discord9@163.com>

* test: cover merge sort optimizer hooks

Signed-off-by: discord9 <discord9@163.com>

* fix: preserve merge sort child topk

Signed-off-by: discord9 <discord9@163.com>

* test: update order by topk plan

Signed-off-by: discord9 <discord9@163.com>

* fix: recognize merge sort global fetch

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-14 12:53:07 +00:00
Weny Xu 7039e54835 fix: require metasrv GC for repartition (#8497) 
* fix: require GC for repartition

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: update sqlness result

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: enable GC for repartition integration tests

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-07-14 12:04:03 +00:00
shuiyisong d667728fde chore!: update promql-parser to v0.10.0, remove holt_winters (#8457) 
* chore: update promql parser and fix compile

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: sqlness

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: cr issue

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: cr issue

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

---------

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
 2026-07-13 03:27:06 +00:00
discord9 b411c6265c test: add json2 variant payload compat case (#8459) 
* test: add json2 variant payload compat case

Signed-off-by: discord9 <discord9@163.com>

* test: refine json2 variant compat case

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-13 02:46:33 +00:00
discord9 02283b6ba0 test(perf): add remote write storage inspection (#8444) 
* feat: add remote write value distributions

Signed-off-by: discord9 <discord9@163.com>

* test: extend remote write value distributions

Signed-off-by: discord9 <discord9@163.com>

* test: inspect remote write parquet storage

Signed-off-by: discord9 <discord9@163.com>

* test: normalize remote write perf fixtures

Signed-off-by: discord9 <discord9@163.com>

* test: add heavy remote write perf case

Signed-off-by: discord9 <discord9@163.com>

* test: use head greptime for read bench

Signed-off-by: discord9 <discord9@163.com>

* test: keep heavy remote write case local

Signed-off-by: discord9 <discord9@163.com>

* test: tune remote write perf smoke case

Signed-off-by: discord9 <discord9@163.com>

* test: fix query fixture import style

Signed-off-by: discord9 <discord9@163.com>

* test: cover remote write value distributions

Signed-off-by: discord9 <discord9@163.com>

* test: add integer counter perf case

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-10 12:27:27 +00:00
jeremyhi 03b304522f refactor: model distributed inspect fan-in as exec (#8447) 
* refactor: model distributed inspect fan-in as exec

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: harden distributed inspect exec fan-in

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: show distributed inspect scan in plans

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: stabilize distributed inspect sqlness

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: apply sqlness replacement to inspect explain

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: redact inspect repartition plan

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-07-10 08:53:23 +00:00
discord9 ff2fa71d50 fix: repartition subset partition key joins (#8460) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-10 06:10:28 +00:00
LFC 6ae687dc8e refactor: optimize json2 write (#8393) 
refactor: optimize json2 write performance

Signed-off-by: luofucong <luofc@foxmail.com>
 2026-07-09 07:08:57 +00:00
discord9 81e32eb1b5 test: add auto flush interval compat case (#8448) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-09 03:06:14 +00:00
fys 3bf8b9722d feat(json2): validate append mode for tables with JSON2 columns (#8434) 
* feat: validate append mode for JSON2 tables

* fix: cr by codex

* fix: unit test

* fix: validate JSON2 append_mode for all ALTER TABLE operations
 2026-07-08 03:33:36 +00:00
discord9 12f83828b1 feat: add Prom remote-write query regression scenario (#8413) 
* feat: add Prom remote-write query regression scenario

Signed-off-by: discord9 <discord9@163.com>

* test: add high-cardinality remote-write query case

Signed-off-by: discord9 <discord9@163.com>

* feat: chunk remote-write query regression loads

Signed-off-by: discord9 <discord9@163.com>

* test: use multi-day remote-write regression case

Signed-off-by: discord9 <discord9@163.com>

* fix: address query regression review comments

Signed-off-by: discord9 <discord9@163.com>

* ci: allow large query regression comments

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-07 07:58:56 +00:00
sribatsha dash 8f5d681e4b feat: support ALTER TABLE SET auto_flush_interval (#8403) 
* feat: support ALTER TABLE SET auto_flush_interval

Closes #8394.

Add a new SetRegionOption::AutoFlushInterval variant so that the
per-table auto flush interval can be changed on existing tables
via 'ALTER TABLE t SET ...', following up the CREATE TABLE path
from #8357.

- region_request.rs: parse 'auto_flush_interval' with humantime and
  map it to the new variant.
- metadata.rs: persist the value (or remove it, if None) in
  TableOptions.extra_options using the same humantime string format
  the engine already expects.
- handle_alter.rs: apply the new interval in
  handle_alter_region_options_fast (no memtable flush needed, same
  pattern as Ttl) and group the variant with Ttl/Twsc in
  new_region_options_on_empty_memtable.

Tests:
- Two unit tests in metadata.rs covering set and unset-to-None.
- A new sqlness case alter_auto_flush_interval.sql covering
  create-then-alter, alter-then-alter, invalid duration, and
  alter on a table that already had auto_flush_interval at create
  time.

Signed-off-by: srivtx <crypticcc101@gmail.com>

* fix: validate auto_flush_interval > 0 in ALTER SET path

Gemini code assist flagged that the request parser accepted a zero
duration, leaving the rejection to the downstream RegionOptions
validation which only fires on next flush. Reject it at parse time
so users get the error immediately at the ALTER TABLE statement.

Also add a '0s' error case to the sqlness test.

Signed-off-by: srivtx <crypticcc101@gmail.com>

* fix: handle SET 'auto_flush_interval' = NULL and add checked-in .result

Address the rest of fengjiachun's review on #8403:

1. Empty value in ALTER SET clears the override (parallels Ttl).
   'ALTER TABLE t SET ... = NULL' comes through as value = ''; we
   now return AutoFlushInterval(None) so the override is removed
   from TableOptions.extra_options, matching the Ttl pattern.

2. Add a unit test in region_request.rs covering the four cases
   (valid, empty-clears, zero-rejected, garbage-rejected).

3. Generate and check in alter_auto_flush_interval.result via
   'cargo sqlness-runner bare -t alter_auto_flush_interval'. Both
   the standalone and distributed sqlness jobs now pass locally,
   and the test extension covers the NULL-clears path end to end.

Signed-off-by: srivtx <crypticcc101@gmail.com>

---------

Signed-off-by: srivtx <crypticcc101@gmail.com>
 2026-07-07 07:34:05 +00:00
fys 63cc232339 chore: add sqlness regression for timestamp precision comparison (#8424) 
* test: add sqlness regression for timestamp precision comparison

* add result
 2026-07-07 06:45:55 +00:00
discord9 c8c7e77cc3 test: add ssts limit compat case (#8427) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-07 03:41:22 +00:00
discord9 1d819a3982 test: add comment metadata compat case (#8402) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-07 02:27:55 +00:00
jeremyhi 5bcc2e9d6b fix: global limit for distributed inspect streams (#8412) 
* fix: global limit for distributed inspect streams

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: honor tighter filter fetch under global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: restore distribution after global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: preserve inherited distribution for global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: propagate inherited hash partitioning

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-07-06 07:02:54 +00:00
discord9 4f5dccf6bb fix(query): push down PromQL cast filters (#8407) 
* fix(query): push down PromQL cast filters

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover cast preimage pushdown

Signed-off-by: discord9 <discord9@163.com>

* test(query): expand cast preimage coverage

Signed-off-by: discord9 <discord9@163.com>

* test(query): compare cast normalization rules

Signed-off-by: discord9 <discord9@163.com>

* test(query): update cast pushdown sqlness plans

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-03 09:10:00 +00:00
discord9 c44f8da646 feat: add query regression perf harness (#8406) 
* feat: add query regression perf harness

Signed-off-by: discord9 <discord9@163.com>

* feat: extend query regression cases

Signed-off-by: discord9 <discord9@163.com>

* ci: harden query regression workflows

Signed-off-by: discord9 <discord9@163.com>

* fix: address query regression review comments

Signed-off-by: discord9 <discord9@163.com>

* ci: limit query regression PR triggers

Signed-off-by: discord9 <discord9@163.com>

* ci: run full query regression case set

Signed-off-by: discord9 <discord9@163.com>

* refactor: model query regression scenarios

Signed-off-by: discord9 <discord9@163.com>

* fix: avoid unenforced query regression thresholds

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-03 09:09:01 +00:00
discord9 424e400c80 test: add flow scheduled now compat case (#8400) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-03 06:31:40 +00:00
Weny Xu 2dc6addb66 feat: report region query stats in heartbeat (#8401) 
* feat: report region read load in heartbeat

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: expose region query stats in information schema

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update sqlness result

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: record region query stats on stream drop

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: keep region query cpu stats in nanoseconds

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-07-03 06:11:57 +00:00
jeremyhi fc7a9bf56d fix: handle PromQL time binary aggregation (#8398) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-07-02 11:18:58 +00:00
discord9 6b587eb0a5 test: add recent sqlness compat cases (#8395) 
* test: add recent sqlness compat cases

Signed-off-by: discord9 <discord9@163.com>

* test: add more sqlness compat cases

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-02 03:13:53 +00:00
discord9 5f6b88b782 fix(flow): bind scheduled now in dist plan (#8389) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-01 10:56:30 +00:00
discord9 88d20d8c93 test: choose compat runner cli args by version (#8376) 
* test: use legacy rpc args in compat runner

Signed-off-by: discord9 <discord9@163.com>

* test: detect compat runner grpc cli args

Signed-off-by: discord9 <discord9@163.com>

* ci: run compat on ready pr open

Signed-off-by: discord9 <discord9@163.com>

* test: choose compat cli args by version

Signed-off-by: discord9 <discord9@163.com>

* test: address compat arg test review

Signed-off-by: discord9 <discord9@163.com>

* test: fix compat runner comment typo

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-01 07:29:32 +00:00
fys 1fea548c17 feat(json2): reject non-object JSON values on write (#8381) 
* feat: disable non object json write

* remove unused code

* fix: sqlness test

* simply code

* add comment

* remove unused unit test

* fix: cargo fmt

* fix: unit test

* fix: unit test
 2026-07-01 03:12:09 +00:00
fys aa563628fb feat(json2): type hint (#8247) 
* feat(json2): type hint

* test(datatypes): add JsonSettings serde round-trip test

* minor refactor

This reverts commit 7ff5a5249a09be5396536284fe822b5761ef4e6a.

* fix: code review
 2026-06-30 07:37:31 +00:00
discord9 d3cc1b1888 feat(flow): stabilize eval interval scheduling (#8360) 
* feat(flow): stabilize eval interval scheduling

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): satisfy eval schedule clippy

Signed-off-by: discord9 <discord9@163.com>

* test(flow): trim eval schedule coverage

Signed-off-by: discord9 <discord9@163.com>

* test(flow): cover stable eval scheduling

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): reserve scheduled runtime hint

Signed-off-by: discord9 <discord9@163.com>

* test(flow): trim sqlness result eof

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): harden eval schedule edges

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): address scheduled flow review

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): clean scheduled config handling

Signed-off-by: discord9 <discord9@163.com>

* test(flow): add eval interval compat case

Signed-off-by: discord9 <discord9@163.com>

* test(flow): cover show create flow in compat

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): drop scheduled time from flow context

Signed-off-by: discord9 <discord9@163.com>

* test(flow): assert scheduled now binding

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-29 09:55:27 +00:00
discord9 16b60fe847 ci: add sqlness compat smoke (#8370) 
* ci: add sqlness compat smoke

Signed-off-by: discord9 <discord9@163.com>

* ci: limit compat smoke pull request trigger

Signed-off-by: discord9 <discord9@163.com>

* ci: run compat smoke in merge queue

Signed-off-by: discord9 <discord9@163.com>

* ci: configure compat version window

Signed-off-by: discord9 <discord9@163.com>

* ci: move compat smoke logic into script

Signed-off-by: discord9 <discord9@163.com>

* ci: expand compat version window

Signed-off-by: discord9 <discord9@163.com>

* ci: rename compat job for recent releases

Signed-off-by: discord9 <discord9@163.com>

* ci: report compat test failures

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-29 08:18:28 +00:00
shuiyisong 59d40c39b1 feat: prw_v2 initial commit with sample ingestion (#8361) 
* chore: add v2 entrance

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: decode request

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: implement remote write v2 for samples

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: add tests

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: remove hand-written proto

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: refactor

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: CR issue

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: CR issue

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: merge tests

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: add source version field

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: fix CR issues

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: test

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: sqlness

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: update proto

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: update proto

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

---------

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
 2026-06-29 03:15:41 +00:00
discord9 16a2b18c2b test: add sqlness compatibility runner (#8334) 
* test: add sqlness compatibility runner

Signed-off-by: discord9 <discord9@163.com>

* fix: update remote dyn filter request

Signed-off-by: discord9 <discord9@163.com>

* fix: satisfy compat runner clippy

Signed-off-by: discord9 <discord9@163.com>

* test: add legacy jsonb compatibility case

Signed-off-by: discord9 <discord9@163.com>

* test: align legacy jsonb compat assertion

Signed-off-by: discord9 <discord9@163.com>

* fix: align remote dyn filter proto fields

Signed-off-by: discord9 <discord9@163.com>

* test: simplify compat topology and SQL handling

Signed-off-by: discord9 <discord9@163.com>

* fix: switch compat runner to target binary

Signed-off-by: discord9 <discord9@163.com>

* test: drop compat SQLNESS comment handling

Signed-off-by: discord9 <discord9@163.com>

* test: support sqlness commands in compat

Signed-off-by: discord9 <discord9@163.com>

* test: fix compat protocol reconnect

Signed-off-by: discord9 <discord9@163.com>

* test: filter compat cases by version

Signed-off-by: discord9 <discord9@163.com>

* test: add merge mode compat case

Signed-off-by: discord9 <discord9@163.com>

* test: harden compat runner setup

Signed-off-by: discord9 <discord9@163.com>

* test: address compat review follow-ups

Signed-off-by: discord9 <discord9@163.com>

* test: tighten compat case validation

Signed-off-by: discord9 <discord9@163.com>

* test: require explicit compat expectations

Signed-off-by: discord9 <discord9@163.com>

* test: generate missing compat snapshots

Signed-off-by: discord9 <discord9@163.com>

* test: add compat dry run

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-26 04:15:27 +00:00
discord9 9cf071808a fix(query): run optimizer rules before MergeScan (#8339) 
* fix(query): push down join filters before MergeScan

Signed-off-by: discord9 <discord9@163.com>

* fix(query): run optimizer before MergeScan pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): narrow pre-MergeScan filter pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): refine pre-MergeScan optimizer prepass

Signed-off-by: discord9 <discord9@163.com>

* fix(query): satisfy predicate extractor clippy

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover pre-MergeScan optimizer edges

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover set comparison prepass

Signed-off-by: discord9 <discord9@163.com>

* fix(query): guard remote scan filter pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): preserve subquery planning errors

Signed-off-by: discord9 <discord9@163.com>

* fix(query): preserve usable scan predicates

Signed-off-by: discord9 <discord9@163.com>

* fix(query): simplify scan predicate extraction

Signed-off-by: discord9 <discord9@163.com>

* fix(query): keep scan filter extraction scoped

Signed-off-by: discord9 <discord9@163.com>

* docs(query): explain pre-MergeScan optimizer

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-23 12:15:40 +00:00
Weny Xu b01ee594f3 feat: add repartition column hint (#8291) 
* feat: add repartition column hint option

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: support altering repartition column hint

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update sqlness result

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: reject time index repartition hint

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: treat rename table as metadata-only alter

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-06-16 09:04:52 +00:00
discord9 f6f56e6bc5 fix!: fence scoped flow repair snapshots (#8277) 
* fix: fence scoped flow repair snapshots

Signed-off-by: discord9 <discord9@163.com>

* test: trim duplicate flow task tests

Signed-off-by: discord9 <discord9@163.com>

* test: moreless

Signed-off-by: discord9 <discord9@163.com>

* refactor: simplify flow query failure fallback

Signed-off-by: discord9 <discord9@163.com>

* test: update need eval interval

Signed-off-by: discord9 <discord9@163.com>

* docs: for helper fn

Signed-off-by: discord9 <discord9@163.com>

* chore: less test

Signed-off-by: discord9 <discord9@163.com>

* refactor: per review

Signed-off-by: discord9 <discord9@163.com>

* test: update for eval interval

Signed-off-by: discord9 <discord9@163.com>

* fix: consume dirty windows after successful query

Signed-off-by: discord9 <discord9@163.com>

* test: rm useless tests

Signed-off-by: discord9 <discord9@163.com>

* test: standalone seq&rm dead if

Signed-off-by: discord9 <discord9@163.com>

* chore: move to pending window instead

Signed-off-by: discord9 <discord9@163.com>

* chore: mark full also call abandon_fenced_repair

Signed-off-by: discord9 <discord9@163.com>

* chore: instant for pending fenced repair

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-12 13:27:12 +00:00