fengyue-xve
1e29e8d4b6
* fix(meta): validate persisted partition bounds before altering regions
A partition bound is persisted with the type of the column at the time
it was written. When an ALTER TABLE changes the type of a column
referenced by a partition expression (widening the time index unit is
the common case), each region rewrites the bound literal to the new
type while building its scan predicate. A millisecond bound like year
3000 fits milliseconds but overflows nanoseconds, so every scan of the
region fails once the table schema is updated -- and some regions may
already be altered when the failure surfaces.
Validate every region's persisted expression on prepare, before any
region is modified: parse the persisted JSON (mirrored types, since
common-meta cannot depend on the partition crate which depends on it),
replay the same value -> ScalarValue conversion and checked cast the
region performs, and reject the ALTER naming the offending region when
a bound cannot be represented with the new column type. Incompatible
changes are rejected atomically; compatible widening proceeds unchanged.
Adds meta-level tests (multi-region rejection naming the second region,
compatible widening completing) and an end-to-end sqlness case.
Fixes #9454
Signed-off-by: fengyue-xve <1759519137@qq.com>
* fix: access ColumnSchema fields directly and use plain format args
ColumnSchema exposes name and data_type as public fields, not methods,
and the telemetry macros only accept a format string after the error.
Signed-off-by: fengyue-xve <1759519137@qq.com>
* test: access ColumnSchema data_type field in alter table tests
Same fix as the previous commit: data_type is a public field on
ColumnSchema, not a method.
Signed-off-by: fengyue-xve <1759519137@qq.com>
* test: drop borrows now that data_type is read as a field
Signed-off-by: fengyue-xve <1759519137@qq.com>
* test: drop useless format! in nested expression test
Signed-off-by: fengyue-xve <1759519137@qq.com>
* test: cover the partition tail in change_col_type sqlness case
Signed-off-by: fengyue-xve <1759519137@qq.com>
* test: drop spurious blank line in change_col_type result
Signed-off-by: fengyue-xve <1759519137@qq.com>
* fix(meta): compare partition bounds with the scan comparison coercion
The preflight validator cast the persisted bound directly to the altered
column type, which rejected ALTERs the scan itself accepts: the region
coerces both the column and the bound to the common comparison type
(ExprSimplifier::coerce in PredicateGroup::new), so narrowing a field
column with an out-of-range bound still scans. Compare through
comparison_coercion(target_type, bound_type) instead and only require the
bound to be castable to the common type.
Also add a round-trip test in the partition crate that serializes real
PartitionExprs and runs them through the validator, so drift between the
Persisted* mirror and the partition types fails CI, and fix the error
message to match the comparison semantics.
Signed-off-by: fengyue-xve <1759519137@qq.com>
* test: update the sqlness fixture for the new incompatibility message
The error message now reflects the comparison semantics: the persisted
bound can no longer be compared with the new column type.
Signed-off-by: fengyue-xve <1759519137@qq.com>
* build(deps): refresh Cargo.lock for the common-meta datafusion-expr dependency
common-meta gained a datafusion-expr dependency in ff9c3f4 but the lock
file was not refreshed, so cargo check --locked refused to run on CI.
Signed-off-by: fengyue-xve <1759519137@qq.com>
* fix(meta): reject every non-widening type change of a partitioned column
The region routes rows on write through `MultiDimPartitionRule::find_region`,
which compares the persisted bound against row values directly: `Value::cmp`
panics on mismatched variants and equality misroutes silently. After a column
partitioned at `v < 3000000000` is altered from BIGINT to INT, every INSERT
compares `Value::Int32` rows against the persisted `Value::Int64` bound and
panics in the frontend, even though the scan-side coercion this check mirrors
still works.
Reject any type change of a column referenced by a persisted partition
expression unless it is a timestamp unit widening, which keeps the bound and
the rows comparable because `Timestamp` comparisons convert units first; the
widening keeps the scan cast check unchanged.
Signed-off-by: fengyue-xve <1759519137@qq.com>
---------
Signed-off-by: fengyue-xve <1759519137@qq.com>
Sqlness Test
Sqlness manual
Case file
Sqlness has two types of file:
.sql: test input, SQL only.result: expected test output, SQL and its results
.result is the output (execution result) file. If you see .result files is changed,
it means this test gets a different result and indicates it fails. You should
check change logs to solve the problem.
You only need to write test SQL in .sql file, and run the test.
Case organization
The root dir of input cases is tests/cases. It contains several subdirectories stand for different test
modes. E.g., standalone/ contains all the tests to run under greptimedb standalone start mode.
Under the first level of subdirectory (e.g. the cases/standalone), you can organize your cases as you like.
Sqlness walks through every file recursively and runs them.
Kafka WAL
Sqlness supports Kafka WAL. You can either provide a Kafka cluster or let sqlness to start one for you.
To run test with kafka, you need to pass the option -w kafka. If no other options are provided, sqlness will use conf/kafka-cluster.yml to start a Kafka cluster. This requires docker and docker-compose commands in your environment.
Otherwise, you can additionally pass the your existing kafka environment to sqlness with -k option. E.g.:
cargo sqlness bare -w kafka -k localhost:9092
In this case, sqlness will not start its own kafka cluster and the one you provided instead.
Run the test
Unlike other tests, this harness is in a binary target form. You can run it with:
cargo sqlness bare
It automatically finishes the following procedures: compile GreptimeDB, start it, grab tests and feed it to
the server, then collect and compare the results. You only need to check if the .result files are changed.
If not, congratulations, the test is passed 🥳!