* fix(query): keep INSERT timestamp conversion out of the source query
Interpreting an INSERT's string timestamps used to work by pushing the
conversion down into the source query, which changed what that query
means. Two consequences:
- Pushing through a UNION's DISTINCT moved the dedup key from the raw
strings to parsed instants, so rows spelling the same instant
differently collapsed into one. On an append-only table that is a
silently dropped row.
- A UNION branch that needed no conversion (a NULL, or an explicit cast)
made the whole column give up, leaving sibling branches on UTC while
the rest of the row used the session timezone.
Convert at the assignment instead, by routing its cast through a
timezone-carrying timestamp type and back. Arrow applies the timezone
when a cast target carries one, and stripping it afterwards preserves
the value. The source query is no longer touched, so both cases go away
and the tree-walking rewrite (roughly 160 lines) is deleted.
The rewrite reads source types, so it now runs TypeCoercion first: a
UNION still carries its loose per-branch schema before coercion, and
retargeting a cast whose input later becomes a timestamp would shift the
value rather than reinterpret it.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(query): address review on INSERT assignment rewrite
- Clone the input `Arc` instead of the whole subtree, and only rebuild it
when a `Values` row actually changes.
- Defer cloning the cast source until the literal-folding path has been
ruled out.
- Move the UTC check onto `Timezone::is_utc`, replacing a bare string
compare.
- Cover a prepared `INSERT ... VALUES (?)`: an untyped placeholder types
as `Null`, so the assignment cast is left for parameter substitution.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(query): follow timestamp insert assignment lineage
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(query): fold constant insert timestamp literals at the assignment
Following lineage by retyping the source column changed every output
column that reads it: a string column sharing the literal was silently
rewritten to a formatted timestamp, and a nanosecond column was
truncated to the precision of whichever column was converted first.
Resolve the constant read-only and fold it into the assignment
expression instead, which leaves the source query untouched and also
covers literals behind WHERE, ORDER BY and DISTINCT. VALUES rows and
UNION branches carry per-row values, so they keep the in-place rewrite,
now guarded against columns with more than one consumer.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(query): strengthen insert lineage regression
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* test(query): trim redundant insert coverage
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix(query): rebuild insert unions loosely and cover UNION distinct
Per review: rebuilding a rewritten union with the strict constructor
rejected legal pre-coercion plans whose untouched columns still differ
across branches. Use try_new_with_loose_types, matching the SQL planner.
Distinct::All joins the rewrite passthrough so UNION (distinct) literals
get session-timezone parsing like UNION ALL; deduplication then keys on
parsed instants instead of raw strings. The top-level rewrite path gains
the same single-consumer guard as rewrite_projection for hand-built DML
plans that share a source column between targets.
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* style(query): tighten insert assignment comments
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
---------
Signed-off-by: Dennis Zhuang <killme2008@gmail.com>
* fix unit tests
* fix: sqlness
* fix/default-time-window:
## Add Helper Functions and Enhance Compaction Tests
- **Refactor Compaction Logic**: Introduced helper functions `flush` and `compact` in `compaction_test.rs` to streamline compaction operations.
- **Enhance Compaction Tests**: Added a new test `test_infer_compaction_time_window` in `compaction_test.rs` to verify compaction time window inference.
- **Testing Improvements**: Added `#[cfg(test)]` attribute to `new_multi_partitions` in `time_partition.rs` to ensure it's only included in test builds.
* fix/default-time-window:
- **Refactor `TimePartition` Struct**: Removed unnecessary comments regarding `time_range` in `time_partition.rs`.
- **Enhance `TimePartitions` Functionality**: Added a method `part_duration_or_default` to provide a default partition duration in `time_partition.rs`.
- **Update SQL Test Cases**: Modified SQL operations and expected results in `scan_big_varchar.result` and `scan_big_varchar.sql` to reflect changes in data manipulation logic.
* fix/default-time-window:
### Update Time Partition Default Duration
- **Refactor Default Duration**: Introduced `INITIAL_TIME_WINDOW` constant to define the default time window duration as `Duration::from_days(1)`. This change replaces multiple instances of the hardcoded default duration across the `time_partition.rs` file.
- **Files Affected**: `time_partition.rs`
* fix/default-time-window:
## Update Partition Duration Handling
- **`time_partition.rs`**: Refactored `part_duration` to be non-optional, removing `Option` wrapper. Updated logic to use `unwrap_or` with `INITIAL_TIME_WINDOW` where necessary. Adjusted related methods and tests to accommodate this change.
- **`version.rs` (memtable and region)**: Updated handling of `part_duration` to align with changes in `time_partition.rs`, ensuring consistent use of non-optional `Duration`.
* fix/default-time-window:
### Improve Error Context in `time_partition.rs`
- Enhanced error context message in `time_partition.rs` to provide clearer information on partition time range issues, including bucket size details.
Signed-off-by: Lei, HUANG <lhuang@greptime.com>
---------
Signed-off-by: Lei, HUANG <lhuang@greptime.com>
dennis zhuang
Morax
Lei, HUANG