Commit Graph

10 Commits

Author SHA1 Message Date
discord9 b3faf22290 fix(tests): make two Windows CI failures deterministic (Nightly CI #8837) (#8840)
* fix(tests): reject overlay directories before opening on all platforms

DatanodeOverlay::load() opened the target before checking is_file(). On
Unix, File::open on a directory succeeds and the loader rejects it with
"must be a regular file". On Windows, File::open on a directory fails up
front with "Access is denied", so the type check was never reached and
the rejects_directories_and_parse_errors test failed 4/4 in Nightly CI
(issue #8837).

Check std::fs::metadata before File::open: metadata succeeds on
directories on both platforms, so the error message is now identical
everywhere and the test assertion holds on Windows too.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

* test(datanode): make test_region_error deterministic across platforms

The second phase raced a 100ms mock handle delay against a 200ms
replay_timeout; on busy Windows CI runners the error could land after the
timeout fired, flaking reply.error.is_some() (Nightly CI, issue #8837).
Use a mock handle that returns the error on its first poll with no delay:
the catchup future completes before replay_timeout can ever fire, so the
test no longer depends on wall-clock scheduling.

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>

---------

Signed-off-by: discord9 <55937128+discord9@users.noreply.github.com>
2026-08-11 05:52:31 +00:00
discord9 bb6d55a99f feat: support old-stage datanode config overlays (#8647)
* feat: support old-stage datanode config overlays

Signed-off-by: discord9 <discord9@163.com>

* fix: derive compat overlay policy from WAL config

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-08-07 08:40:46 +00:00
Weny Xu ad2c1bfc59 fix(meta): preserve legacy WAL options compatibility (#8707)
* fix: preserve legacy region WAL options format

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: add downgrade compatibility coverage

Signed-off-by: WenyXu <wenymedia@gmail.com>

* refactor: deduplicate compat restart

Signed-off-by: WenyXu <wenymedia@gmail.com>

* test: harden downgrade compatibility check

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2026-07-31 13:40:27 +00:00
jeremyhi 448f973593 fix: sandbox SQL local filesystem access (#8708)
* fix: sandbox SQL local filesystem access

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: address local file sandbox review findings

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: support Windows local copy paths

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: improve sandbox path errors

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* refactor: simplify local path error context

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* perf: stream secure filesystem listings

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* style: derive local file access default

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: improve local file access errors

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: address local file access review findings

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* test: simplify local file access coverage

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: harden sandboxed local file backends

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: reject directory copy targets before creation

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

* fix: avoid implicit string clone in file table listing

Signed-off-by: jeremyhi <fengjiachun@gmail.com>

---------

Signed-off-by: jeremyhi <fengjiachun@gmail.com>
2026-07-31 13:23:15 +00:00
Lei, HUANG 6a53bb053c feat: enable soft-drop table lifecycle (#8554)
* feat: add purge_table admin function

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: restrict purge_table to admin

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: keep purge tombstone lookup consistent

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* test: verify soft-drop table lifecycle

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* ci: run soft-drop lifecycle sqlness

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: read purge tombstones authoritatively

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): invalidate soft-drop marker caches

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* feat(meta): enable configurable table soft drop

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(ci): configure gc for soft-drop sqlness

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): close regions before soft-drop metadata

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): avoid rollback after soft-drop tombstone

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(function): validate admin single-row calls

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): log purge dropped table target

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix(meta): mark soft-drop config experimental

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

* fix: sqlness test base

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>

---------

Signed-off-by: Lei, HUANG <ratuthomm@gmail.com>
2026-07-20 08:28:35 +00:00
discord9 16a2b18c2b test: add sqlness compatibility runner (#8334)
* test: add sqlness compatibility runner

Signed-off-by: discord9 <discord9@163.com>

* fix: update remote dyn filter request

Signed-off-by: discord9 <discord9@163.com>

* fix: satisfy compat runner clippy

Signed-off-by: discord9 <discord9@163.com>

* test: add legacy jsonb compatibility case

Signed-off-by: discord9 <discord9@163.com>

* test: align legacy jsonb compat assertion

Signed-off-by: discord9 <discord9@163.com>

* fix: align remote dyn filter proto fields

Signed-off-by: discord9 <discord9@163.com>

* test: simplify compat topology and SQL handling

Signed-off-by: discord9 <discord9@163.com>

* fix: switch compat runner to target binary

Signed-off-by: discord9 <discord9@163.com>

* test: drop compat SQLNESS comment handling

Signed-off-by: discord9 <discord9@163.com>

* test: support sqlness commands in compat

Signed-off-by: discord9 <discord9@163.com>

* test: fix compat protocol reconnect

Signed-off-by: discord9 <discord9@163.com>

* test: filter compat cases by version

Signed-off-by: discord9 <discord9@163.com>

* test: add merge mode compat case

Signed-off-by: discord9 <discord9@163.com>

* test: harden compat runner setup

Signed-off-by: discord9 <discord9@163.com>

* test: address compat review follow-ups

Signed-off-by: discord9 <discord9@163.com>

* test: tighten compat case validation

Signed-off-by: discord9 <discord9@163.com>

* test: require explicit compat expectations

Signed-off-by: discord9 <discord9@163.com>

* test: generate missing compat snapshots

Signed-off-by: discord9 <discord9@163.com>

* test: add compat dry run

Signed-off-by: discord9 <discord9@163.com>

---------

Signed-off-by: discord9 <discord9@163.com>
2026-06-26 04:15:27 +00:00
Ruihang Xia c9f73b1696 feat: start environments in parallel (#8101)
* start environments in parallel

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use merged commit

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix bins_dir racing

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
2026-05-14 09:44:26 +00:00
Yingwen 233e35c0c9 feat!: switch default sst format to flat (#7909)
* feat: support alter from primary_key to flat

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: alter flat to primary_key

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: change default_experimental_flat_format to true

Signed-off-by: evenyag <realevenyag@gmail.com>

* feat: compute channel size from splitted batch size

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: add tests for split and channel size

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: always set sst_format from manifest on region open

sanitize_region_options did not set options.sst_format when the
default (PrimaryKey) matched the manifest value, leaving it as None
after reopen. This caused the alter format change to appear lost.

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: fix tests

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: show create table after alteration

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor!: rename default_experimental_flat_format to default_flat_format

The flat format is no longer experimental. Remove "experimental" from
the config field name, doc comments, and all references.

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: fix clippy

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2026-04-03 04:14:02 +00:00
Yingwen 0939dc1d32 test: run sqlness for flat format (#7178)
* test: support flat format in sqlness

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: replace region stats test result with NUM

Signed-off-by: evenyag <realevenyag@gmail.com>

* ci: add flat format to sqlness ci

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>
2025-11-05 11:23:12 +00:00
Weny Xu 40e9ce90a7 refactor: restructure sqlness to support multiple envs and extract common utils (#7066)
* refactor: restructure sqlness to support multiple envs and extract common utils

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore(ci): update sqlness cmd

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: add comments

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: error fmt

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: only reconnect mysql and pg client

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
2025-10-11 06:34:17 +00:00