rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-07-09 15:30:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,657 Commits 149 Branches 261 Tags
ebfe4f6eb24555321590f324a9be8cd2cd19f09c
Commit Graph

936 Commits

Author SHA1 Message Date
discord9
12f83828b1 feat: add Prom remote-write query regression scenario (#8413) 
* feat: add Prom remote-write query regression scenario

Signed-off-by: discord9 <discord9@163.com>

* test: add high-cardinality remote-write query case

Signed-off-by: discord9 <discord9@163.com>

* feat: chunk remote-write query regression loads

Signed-off-by: discord9 <discord9@163.com>

* test: use multi-day remote-write regression case

Signed-off-by: discord9 <discord9@163.com>

* fix: address query regression review comments

Signed-off-by: discord9 <discord9@163.com>

* ci: allow large query regression comments

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-07 07:58:56 +00:00
sribatsha dash
8f5d681e4b feat: support ALTER TABLE SET auto_flush_interval (#8403) 
* feat: support ALTER TABLE SET auto_flush_interval

Closes #8394.

Add a new SetRegionOption::AutoFlushInterval variant so that the
per-table auto flush interval can be changed on existing tables
via 'ALTER TABLE t SET ...', following up the CREATE TABLE path
from #8357.

- region_request.rs: parse 'auto_flush_interval' with humantime and
  map it to the new variant.
- metadata.rs: persist the value (or remove it, if None) in
  TableOptions.extra_options using the same humantime string format
  the engine already expects.
- handle_alter.rs: apply the new interval in
  handle_alter_region_options_fast (no memtable flush needed, same
  pattern as Ttl) and group the variant with Ttl/Twsc in
  new_region_options_on_empty_memtable.

Tests:
- Two unit tests in metadata.rs covering set and unset-to-None.
- A new sqlness case alter_auto_flush_interval.sql covering
  create-then-alter, alter-then-alter, invalid duration, and
  alter on a table that already had auto_flush_interval at create
  time.

Signed-off-by: srivtx <crypticcc101@gmail.com>

* fix: validate auto_flush_interval > 0 in ALTER SET path

Gemini code assist flagged that the request parser accepted a zero
duration, leaving the rejection to the downstream RegionOptions
validation which only fires on next flush. Reject it at parse time
so users get the error immediately at the ALTER TABLE statement.

Also add a '0s' error case to the sqlness test.

Signed-off-by: srivtx <crypticcc101@gmail.com>

* fix: handle SET 'auto_flush_interval' = NULL and add checked-in .result

Address the rest of fengjiachun's review on #8403:

1. Empty value in ALTER SET clears the override (parallels Ttl).
   'ALTER TABLE t SET ... = NULL' comes through as value = ''; we
   now return AutoFlushInterval(None) so the override is removed
   from TableOptions.extra_options, matching the Ttl pattern.

2. Add a unit test in region_request.rs covering the four cases
   (valid, empty-clears, zero-rejected, garbage-rejected).

3. Generate and check in alter_auto_flush_interval.result via
   'cargo sqlness-runner bare -t alter_auto_flush_interval'. Both
   the standalone and distributed sqlness jobs now pass locally,
   and the test extension covers the NULL-clears path end to end.

Signed-off-by: srivtx <crypticcc101@gmail.com>

---------

Signed-off-by: srivtx <crypticcc101@gmail.com>
 2026-07-07 07:34:05 +00:00
fys
63cc232339 chore: add sqlness regression for timestamp precision comparison (#8424) 
* test: add sqlness regression for timestamp precision comparison

* add result
 2026-07-07 06:45:55 +00:00
discord9
c8c7e77cc3 test: add ssts limit compat case (#8427) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-07 03:41:22 +00:00
discord9
1d819a3982 test: add comment metadata compat case (#8402) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-07 02:27:55 +00:00
jeremyhi
5bcc2e9d6b fix: global limit for distributed inspect streams (#8412) 
* fix: global limit for distributed inspect streams

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: honor tighter filter fetch under global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: restore distribution after global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: preserve inherited distribution for global limit

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: propagate inherited hash partitioning

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-07-06 07:02:54 +00:00
discord9
4f5dccf6bb fix(query): push down PromQL cast filters (#8407) 
* fix(query): push down PromQL cast filters

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover cast preimage pushdown

Signed-off-by: discord9 <discord9@163.com>

* test(query): expand cast preimage coverage

Signed-off-by: discord9 <discord9@163.com>

* test(query): compare cast normalization rules

Signed-off-by: discord9 <discord9@163.com>

* test(query): update cast pushdown sqlness plans

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-03 09:10:00 +00:00
discord9
c44f8da646 feat: add query regression perf harness (#8406) 
* feat: add query regression perf harness

Signed-off-by: discord9 <discord9@163.com>

* feat: extend query regression cases

Signed-off-by: discord9 <discord9@163.com>

* ci: harden query regression workflows

Signed-off-by: discord9 <discord9@163.com>

* fix: address query regression review comments

Signed-off-by: discord9 <discord9@163.com>

* ci: limit query regression PR triggers

Signed-off-by: discord9 <discord9@163.com>

* ci: run full query regression case set

Signed-off-by: discord9 <discord9@163.com>

* refactor: model query regression scenarios

Signed-off-by: discord9 <discord9@163.com>

* fix: avoid unenforced query regression thresholds

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-03 09:09:01 +00:00
discord9
424e400c80 test: add flow scheduled now compat case (#8400) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-03 06:31:40 +00:00
Weny Xu
2dc6addb66 feat: report region query stats in heartbeat (#8401) 
* feat: report region read load in heartbeat

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: expose region query stats in information schema

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update sqlness result

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: record region query stats on stream drop

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: keep region query cpu stats in nanoseconds

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-07-03 06:11:57 +00:00
jeremyhi
fc7a9bf56d fix: handle PromQL time binary aggregation (#8398) 
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
 2026-07-02 11:18:58 +00:00
discord9
6b587eb0a5 test: add recent sqlness compat cases (#8395) 
* test: add recent sqlness compat cases

Signed-off-by: discord9 <discord9@163.com>

* test: add more sqlness compat cases

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-02 03:13:53 +00:00
discord9
5f6b88b782 fix(flow): bind scheduled now in dist plan (#8389) 
Signed-off-by: discord9 <discord9@163.com>
 2026-07-01 10:56:30 +00:00
discord9
88d20d8c93 test: choose compat runner cli args by version (#8376) 
* test: use legacy rpc args in compat runner

Signed-off-by: discord9 <discord9@163.com>

* test: detect compat runner grpc cli args

Signed-off-by: discord9 <discord9@163.com>

* ci: run compat on ready pr open

Signed-off-by: discord9 <discord9@163.com>

* test: choose compat cli args by version

Signed-off-by: discord9 <discord9@163.com>

* test: address compat arg test review

Signed-off-by: discord9 <discord9@163.com>

* test: fix compat runner comment typo

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-07-01 07:29:32 +00:00
fys
1fea548c17 feat(json2): reject non-object JSON values on write (#8381) 
* feat: disable non object json write

* remove unused code

* fix: sqlness test

* simply code

* add comment

* remove unused unit test

* fix: cargo fmt

* fix: unit test

* fix: unit test
 2026-07-01 03:12:09 +00:00
fys
aa563628fb feat(json2): type hint (#8247) 
* feat(json2): type hint

* test(datatypes): add JsonSettings serde round-trip test

* minor refactor

This reverts commit 7ff5a5249a09be5396536284fe822b5761ef4e6a.

* fix: code review
 2026-06-30 07:37:31 +00:00
discord9
d3cc1b1888 feat(flow): stabilize eval interval scheduling (#8360) 
* feat(flow): stabilize eval interval scheduling

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): satisfy eval schedule clippy

Signed-off-by: discord9 <discord9@163.com>

* test(flow): trim eval schedule coverage

Signed-off-by: discord9 <discord9@163.com>

* test(flow): cover stable eval scheduling

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): reserve scheduled runtime hint

Signed-off-by: discord9 <discord9@163.com>

* test(flow): trim sqlness result eof

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): harden eval schedule edges

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): address scheduled flow review

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): clean scheduled config handling

Signed-off-by: discord9 <discord9@163.com>

* test(flow): add eval interval compat case

Signed-off-by: discord9 <discord9@163.com>

* test(flow): cover show create flow in compat

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): drop scheduled time from flow context

Signed-off-by: discord9 <discord9@163.com>

* test(flow): assert scheduled now binding

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-29 09:55:27 +00:00
discord9
16b60fe847 ci: add sqlness compat smoke (#8370) 
* ci: add sqlness compat smoke

Signed-off-by: discord9 <discord9@163.com>

* ci: limit compat smoke pull request trigger

Signed-off-by: discord9 <discord9@163.com>

* ci: run compat smoke in merge queue

Signed-off-by: discord9 <discord9@163.com>

* ci: configure compat version window

Signed-off-by: discord9 <discord9@163.com>

* ci: move compat smoke logic into script

Signed-off-by: discord9 <discord9@163.com>

* ci: expand compat version window

Signed-off-by: discord9 <discord9@163.com>

* ci: rename compat job for recent releases

Signed-off-by: discord9 <discord9@163.com>

* ci: report compat test failures

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-29 08:18:28 +00:00
shuiyisong
59d40c39b1 feat: prw_v2 initial commit with sample ingestion (#8361) 
* chore: add v2 entrance

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: decode request

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: implement remote write v2 for samples

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: add tests

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: remove hand-written proto

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: refactor

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: CR issue

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: CR issue

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: merge tests

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: add source version field

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: fix CR issues

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: test

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* fix: sqlness

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: update proto

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

* chore: update proto

Signed-off-by: shuiyisong <xixing.sys@gmail.com>

---------

Signed-off-by: shuiyisong <xixing.sys@gmail.com>
 2026-06-29 03:15:41 +00:00
discord9
16a2b18c2b test: add sqlness compatibility runner (#8334) 
* test: add sqlness compatibility runner

Signed-off-by: discord9 <discord9@163.com>

* fix: update remote dyn filter request

Signed-off-by: discord9 <discord9@163.com>

* fix: satisfy compat runner clippy

Signed-off-by: discord9 <discord9@163.com>

* test: add legacy jsonb compatibility case

Signed-off-by: discord9 <discord9@163.com>

* test: align legacy jsonb compat assertion

Signed-off-by: discord9 <discord9@163.com>

* fix: align remote dyn filter proto fields

Signed-off-by: discord9 <discord9@163.com>

* test: simplify compat topology and SQL handling

Signed-off-by: discord9 <discord9@163.com>

* fix: switch compat runner to target binary

Signed-off-by: discord9 <discord9@163.com>

* test: drop compat SQLNESS comment handling

Signed-off-by: discord9 <discord9@163.com>

* test: support sqlness commands in compat

Signed-off-by: discord9 <discord9@163.com>

* test: fix compat protocol reconnect

Signed-off-by: discord9 <discord9@163.com>

* test: filter compat cases by version

Signed-off-by: discord9 <discord9@163.com>

* test: add merge mode compat case

Signed-off-by: discord9 <discord9@163.com>

* test: harden compat runner setup

Signed-off-by: discord9 <discord9@163.com>

* test: address compat review follow-ups

Signed-off-by: discord9 <discord9@163.com>

* test: tighten compat case validation

Signed-off-by: discord9 <discord9@163.com>

* test: require explicit compat expectations

Signed-off-by: discord9 <discord9@163.com>

* test: generate missing compat snapshots

Signed-off-by: discord9 <discord9@163.com>

* test: add compat dry run

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-26 04:15:27 +00:00
discord9
9cf071808a fix(query): run optimizer rules before MergeScan (#8339) 
* fix(query): push down join filters before MergeScan

Signed-off-by: discord9 <discord9@163.com>

* fix(query): run optimizer before MergeScan pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): narrow pre-MergeScan filter pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): refine pre-MergeScan optimizer prepass

Signed-off-by: discord9 <discord9@163.com>

* fix(query): satisfy predicate extractor clippy

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover pre-MergeScan optimizer edges

Signed-off-by: discord9 <discord9@163.com>

* test(query): cover set comparison prepass

Signed-off-by: discord9 <discord9@163.com>

* fix(query): guard remote scan filter pushdown

Signed-off-by: discord9 <discord9@163.com>

* fix(query): preserve subquery planning errors

Signed-off-by: discord9 <discord9@163.com>

* fix(query): preserve usable scan predicates

Signed-off-by: discord9 <discord9@163.com>

* fix(query): simplify scan predicate extraction

Signed-off-by: discord9 <discord9@163.com>

* fix(query): keep scan filter extraction scoped

Signed-off-by: discord9 <discord9@163.com>

* docs(query): explain pre-MergeScan optimizer

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-23 12:15:40 +00:00
Weny Xu
b01ee594f3 feat: add repartition column hint (#8291) 
* feat: add repartition column hint option

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: support altering repartition column hint

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: update sqlness result

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: reject time index repartition hint

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: treat rename table as metadata-only alter

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-06-16 09:04:52 +00:00
discord9
f6f56e6bc5 fix!: fence scoped flow repair snapshots (#8277) 
* fix: fence scoped flow repair snapshots

Signed-off-by: discord9 <discord9@163.com>

* test: trim duplicate flow task tests

Signed-off-by: discord9 <discord9@163.com>

* test: moreless

Signed-off-by: discord9 <discord9@163.com>

* refactor: simplify flow query failure fallback

Signed-off-by: discord9 <discord9@163.com>

* test: update need eval interval

Signed-off-by: discord9 <discord9@163.com>

* docs: for helper fn

Signed-off-by: discord9 <discord9@163.com>

* chore: less test

Signed-off-by: discord9 <discord9@163.com>

* refactor: per review

Signed-off-by: discord9 <discord9@163.com>

* test: update for eval interval

Signed-off-by: discord9 <discord9@163.com>

* fix: consume dirty windows after successful query

Signed-off-by: discord9 <discord9@163.com>

* test: rm useless tests

Signed-off-by: discord9 <discord9@163.com>

* test: standalone seq&rm dead if

Signed-off-by: discord9 <discord9@163.com>

* chore: move to pending window instead

Signed-off-by: discord9 <discord9@163.com>

* chore: mark full also call abandon_fenced_repair

Signed-off-by: discord9 <discord9@163.com>

* chore: instant for pending fenced repair

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-12 13:27:12 +00:00
discord9
7ec89dcf80 test: redact dynamic filter in order_by sqlness (#8286) 
Signed-off-by: discord9 <discord9@163.com>
 2026-06-12 11:14:27 +00:00
discord9
2febea6ec3 test: add rebuild index coverage (#8175) 
* test: basic rebuild index test

Signed-off-by: discord9 <discord9@163.com>

* test: address rebuild index review comments

Signed-off-by: discord9 <discord9@163.com>

* test: update build index restart expected output

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-11 14:03:24 +00:00
dennis zhuang
ca21c9f048 fix!: correct information_schema index metadata (#8275) 
fix: correct information_schema index metadata

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2026-06-11 01:36:23 +00:00
discord9
8fc5a3b1c7 feat: apply remote dynamic filters on datanode scans (#8262) 
* feat: apply rdf

Signed-off-by: discord9 <discord9@163.com>

* chore: clippy

Signed-off-by: discord9 <discord9@163.com>

* fix: drop remote dyn filter fallback exec

Signed-off-by: discord9 <discord9@163.com>

* Revert "fix: drop remote dyn filter fallback exec"

This reverts commit bb757a596c.

Signed-off-by: discord9 <discord9@163.com>

* refactor: use rdf receiver logical plan instead

Signed-off-by: discord9 <discord9@163.com>

* test: update sqlness

Signed-off-by: discord9 <discord9@163.com>

* feat: rdf disable option

Signed-off-by: discord9 <discord9@163.com>

* tests: large int tests

Signed-off-by: discord9 <discord9@163.com>

* chore: clippy

Signed-off-by: discord9 <discord9@163.com>

* refactor: per review

Signed-off-by: discord9 <discord9@163.com>

* test: update prec fix

Signed-off-by: discord9 <discord9@163.com>

* fix: make receiver node works

Signed-off-by: discord9 <discord9@163.com>

* refactor: per review

Signed-off-by: discord9 <discord9@163.com>

* fix: tql disable rdf

Signed-off-by: discord9 <discord9@163.com>

* chore: rm useless joins

Signed-off-by: discord9 <discord9@163.com>

* fix: also disable in flow tql

Signed-off-by: discord9 <discord9@163.com>

* refactor: per review rm to promql

Signed-off-by: discord9 <discord9@163.com>

* chore: promql ut

Signed-off-by: discord9 <discord9@163.com>

* per review

Signed-off-by: discord9 <discord9@163.com>

* test: rm misleading&add some nested/cleanup

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-10 12:57:23 +00:00
dennis zhuang
e403133eb2 feat: add information_schema statistics table (#8253) 
* feat: add information_schema statistics table

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: use index-local sequence in statistics

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: ordinal_position for pk

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* fix: statistics.nullable uses empty string for non-nullable columns

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2026-06-08 12:52:02 +00:00
Ruihang Xia
428cd8137d feat: identify noneffective binary modifiers (#8230) 
* perf(promql): use tsid for full-label modifier joins

* fix(promql): collect narrow range joins by tsid

* verify on for both sides

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* check if tsid exists

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* correct behavior

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-06-05 08:44:50 +00:00
dennis zhuang
d6c37778ae feat: table semantic layer information_schema view (Phase 3) (#8240) 
* feat: table semantic layer information_schema view (Phase 3)

Add `information_schema.table_semantics`, a queryable view over the table
semantic layer. One row per table that carries at least one
`greptime.semantic.*` option: the signal-agnostic keys
(signal_type/source/pipeline/metadata_quality) are promoted to columns and
the remaining signal-specific keys are folded into a `semantic_options`
JSON string. Tables with no semantic key are excluded.

Stacked on Phase 2.

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: address PR review on table_semantics

- fold JSON serialization failure into None instead of unwrap/panic
- drop per-row Vec allocation in predicate eval; use a fixed array
- align RFC view name with the shipped `table_semantics`

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

* chore: update results

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>

---------

Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
 2026-06-05 08:16:41 +00:00
QuakeWang
6ed67167bb feat: support headerless CSV copy from (#8233) 
* feat: support headerless CSV copy from

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>

* fix: update csv copy sqlness result

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>

* test: cover headerless CSV copy from

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>

* test: cover headerless CSV column count mismatch

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>

---------

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
 2026-06-04 12:04:15 +00:00
Ruihang Xia
edd563d2e1 feat: join simplifier for promql binary op (#8211) 
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refactor, remove shallow methods

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* more sqlness cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clear ctx

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* prefix leaf aliase

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-06-04 08:13:24 +00:00
Weny Xu
bb1de2a744 test: move CSV skip bad records coverage to integration (#8237) 
Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-06-04 07:58:58 +00:00
discord9
d304df6e75 fix: run eval-interval flow without time window (#8231) 
* fix: run eval-interval flow without time window

Signed-off-by: discord9 <discord9@163.com>

* test: cover eval-interval flow join query

Signed-off-by: discord9 <discord9@163.com>

* fix: address eval interval flow review comments

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-03 13:06:22 +00:00
QuakeWang
ca07a53deb feat: support CSV copy skip bad records (#8198) 
* feat: support CSV copy headers and skip bad records

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>

* refactor: reuse CSV skippable error helper

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>

* fix: address CSV skip bad records review

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>

* docs: clarify CSV skip bad records performance tradeoff

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>

---------

Signed-off-by: QuakeWang <wangfuzheng0814@foxmail.com>
 2026-06-03 12:11:02 +00:00
discord9
7f76ad5439 feat: validate batching flow sink schema on create (#8176) 
* feat: check schema on create

Signed-off-by: discord9 <discord9@163.com>

* chore: update sqlness

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): avoid duplicate fields when matching sink schema

Signed-off-by: discord9 <discord9@163.com>

* fix: null handling

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* chore: debug log

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-02 08:24:50 +00:00
discord9
28fd796f4e fix(flow): harden incremental read correctness (#8196) 
* fix(flow): harden incremental read correctness

Signed-off-by: discord9 <discord9@163.com>

* fix(flow): propagate dirty window options

Signed-off-by: discord9 <discord9@163.com>

* test: more

Signed-off-by: discord9 <discord9@163.com>

* chore: test config api

Signed-off-by: discord9 <discord9@163.com>

* refactor: split gen

Signed-off-by: discord9 <discord9@163.com>

* chore: per review

Signed-off-by: discord9 <discord9@163.com>

* fix: allowlist key

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-06-01 02:48:00 +00:00
LFC
869a584f8a ci: add nightly jsonbench test (#7750) 
Signed-off-by: luofucong <luofc@foxmail.com>
 2026-05-29 07:07:49 +00:00
discord9
ba15a9c056 feat: support pending flow metadata with defer_on_missing_source (#8124) 
* feat: support defer_on_missing_source for pending flow creation

Add `defer_on_missing_source` flow option that allows creating flows
even when source tables do not yet exist. The flow enters a pending
state and is automatically activated when source tables become available.

Key changes:
- New `FlowStatus::PendingSources` and fields in `FlowInfoValue` for
  unresolved source table names and last activation error
- `defer_on_missing_source` create-time-only option: stripped from
  runtime/flownode `CreateRequest` but preserved in metadata for
  SQL round-trip (`SHOW CREATE FLOW`, `information_schema.flows`)
- `CreateFlowProcedure` creates pending metadata when sources are
  missing and `defer_on_missing_source=true`; falls back to
  `FlowType::Batching` for missing-source flows
- `PendingFlowReconcileManager` in meta-srv periodically checks
  pending flows and activates them when source tables resolve
- `ActivatePendingFlowProcedure` handles activation: allocates peers,
  creates flows on flownodes, updates metadata, invalidates cache
- `OR REPLACE` properly handles pending<->active transitions,
  including peer allocation and flownode flow teardown
- `FlowMetadataAllocator::alloc_peers` for peer allocation at
  activation time
- Validated flow options: only `defer_on_missing_source` allowed;
  unknown options rejected
- Known issue: standalone mode does not support flownodes, so
  pending flow flush/sink behavior covered only in distributed
  sqlness; operator and meta unit tests cover activation logic

Tests:
- operator `determine_flow_type_for_source_state` (3 passed)
- common-meta `create_flow` (19 passed) including replacement
- common-meta `activate_flow` (4 passed)
- meta-srv `flow` (11 passed)
- sqlness: `flow_pending` covers create/replace/round-trip

Signed-off-by: discord9 <discord9@163.com>

* chore: simplify pending flow PR scope

Reduce PR #8124 to the metadata-only MVP after complexity review.

Changes:
- Remove automatic activation procedure and meta-srv reconcile wiring
- Remove activation tests and activation-only metadata fields
- Reject cross-state pending<->active `OR REPLACE` transitions for now
- Keep pending metadata creation and SQL round-trip behavior
- Allow `DROP FLOW` for pending flows without routes
- Reduce flow_pending sqlness to metadata/round-trip/drop coverage only

Deferred follow-ups are documented locally in `.tmp/tasks/pending-defer-semantics/deferred-followups.md` and intentionally not committed.

Tests:
- `cargo test -p operator determine_flow_type_for_source_state`
- `cargo test -p common-meta create_flow`
- `cargo test -p common-meta drop_flow`
- `cargo sqlness bare --test-filter flow_pending --bins-dir /mnt/nvme_rust/rust-targets/pending_defer/debug`

Signed-off-by: discord9 <discord9@163.com>

* test: cover pending flow metadata edge cases

Signed-off-by: discord9 <discord9@163.com>

* test: fix pending flow metadata test lint

Signed-off-by: discord9 <discord9@163.com>

* docs: document pending flow metadata fields

Signed-off-by: discord9 <discord9@163.com>

* chore: more sleep when test

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-05-29 06:59:21 +00:00
Ruihang Xia
85ae29cb0c perf: collect narrow binary join (#8193) 
* perf(promql): collect narrow binary join build side

* fix projection

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* finalize

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename mod

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2026-05-28 11:30:30 +00:00
Lei, HUANG
9a4e5e8457 chore: expose region info inspection table (#8178) 
* chore/region-sync-diff: add region info inspection core

- `store-api`: add `RegionInfoEntry` schema and plan builder in `src/store-api/src/region_info.rs` and export it from `src/store-api/src/lib.rs`
- `mito2`: collect region runtime metadata with `MitoEngine::all_region_infos` and `RegionRoleState::as_str` in `src/mito2/src/engine.rs`, `src/mito2/src/region.rs`, `src/mito2/src/engine/basic_test.rs`, `src/mito2/Cargo.toml`, and `Cargo.lock`
- `datanode`: expose the reserved `InspectRegionInfo` provider in `src/datanode/src/region_server/catalog.rs`

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* chore/region-sync-diff: expose region info schema table

- `information_schema.region_info`: add frontend table wiring in `src/catalog/src/system_schema/information_schema.rs`, `src/catalog/src/system_schema/information_schema/region_info.rs`, `src/catalog/src/system_schema/information_schema/table_names.rs`, and `src/common/catalog/src/consts.rs`
- `region_group` removal: drop `region_group` from `src/store-api/src/region_info.rs`, `src/mito2/src/region.rs`, and `src/mito2/src/engine/basic_test.rs`
- `SQLness coverage`: add standalone coverage in `tests/cases/standalone/common/information_schema/region_info.sql` and `tests/cases/standalone/common/information_schema/region_info.result`

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* chore/region-sync-diff: restore region group info

- `region_info` schema: restore `region_group` alongside `region_sequence` in `src/store-api/src/region_info.rs`, `src/mito2/src/region.rs`, `src/mito2/src/engine/basic_test.rs`, and `tests/cases/standalone/common/information_schema/region_info.result`
- `MitoEngine::all_region_infos`: remove redundant iterator conversion in `src/mito2/src/engine.rs`

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix: sqlness

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* fix: sqlness

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* chore/region-sync-diff: clarify region sequence columns

- `region_info` schema: rename `sequence` to `committed_sequence` and add nullable `flushed_sequence` in `src/store-api/src/region_info.rs` and `src/mito2/src/region.rs`
- `region_info` coverage: update sequence assertions and expected metadata in `src/mito2/src/engine/basic_test.rs`, `tests/cases/standalone/common/information_schema/region_info.sql`, `tests/cases/standalone/common/information_schema/region_info.result`, and `tests/cases/standalone/common/system/information_schema.result`

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

* chore/region-sync-diff: report region options errors

- `region_info` output: preserve `region_options` serialization failures as JSON error objects in `src/mito2/src/region.rs`

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>

---------

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
 2026-05-28 09:52:37 +00:00
discord9
eccd97b5c7 feat(flow): support incremental read checkpoints (#8179) 
* feat: flownode inc mode

Signed-off-by: discord9 <discord9@163.com>

* chore: rename fallback reason

Signed-off-by: discord9 <discord9@163.com>

* fix: harden flow incremental checkpoints

Signed-off-by: discord9 <discord9@163.com>

* fix: address flow watermark lint

Signed-off-by: discord9 <discord9@163.com>

* fix: address flow clippy

Signed-off-by: discord9 <discord9@163.com>

* refactor: clarify incremental plan preparation

Signed-off-by: discord9 <discord9@163.com>

* refactor: per review

Signed-off-by: discord9 <discord9@163.com>

* refactor: per review

Signed-off-by: discord9 <discord9@163.com>

* test: more sqlness test

Signed-off-by: discord9 <discord9@163.com>

* refactor: per review

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
 2026-05-28 09:31:46 +00:00
LFC
bf7e3551fe test: add jsonbench tests (#8165) 
Signed-off-by: luofucong <luofc@foxmail.com>
 2026-05-27 08:34:06 +00:00
Yingwen
9487e2c3ca fix: divide series for subquery output (#8173) 
* fix: divide series for subquery output

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: propagate time index lookup error in prom_call_manipulate

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2026-05-27 07:10:24 +00:00
fys
8d3ebde652 fix(mito2): schema-safe skipping index pruning (#8122) 
* fix: schema-safe skipping index pruning

* fix: cargo clippy

* fix: sqlness test

* remove default plan in BloomFilterIndexApplier

* fix comment of plan_for_sst

* add fast path for default_plan

* minor refactor

* some rename

* fix: cr by ai
 2026-05-25 11:45:42 +00:00
Yingwen
a25152664b fix: qualify HistogramFold schema (#8157) 
* test: add regression test for binary op on histogram_quantile (#8144)

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: preserve column qualifiers in HistogramFold output schema (#8144)

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
 2026-05-25 07:40:48 +00:00
Weny Xu
1cd6b30058 fix: reject physical metric table writes (#8153) 
Signed-off-by: WenyXu <wenymedia@gmail.com>
 2026-05-22 11:57:43 +00:00
LFC
59b738d7f2 feat: compact json2 data (#8103) 
Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 2026-05-21 07:14:32 +00:00
LFC
a6dc55a71c feat: show sst primary key range in information_schema (#8137) 
Signed-off-by: luofucong <luofc@foxmail.com>
 2026-05-20 12:50:18 +00:00
fys
418318da51 fix: flaky sqlness result ordering (#8136) 
* fix: flaky sqlness result ordering

* fix: flaky sqlness result
 2026-05-19 16:29:21 +00:00