feat: Implements last-non-null dedup strategy for flat format (#6709)

Browse Source

* feat: Implements FlatLastNonNull strategy

Dedup rows and keep last non null fields

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: add basic test for FlatLastNonNull

Signed-off-by: evenyag <realevenyag@gmail.com>

* test: port more last non null test

Signed-off-by: evenyag <realevenyag@gmail.com>

* fix: do merge rows after delete op

Signed-off-by: evenyag <realevenyag@gmail.com>

* style: fix clippy

Signed-off-by: evenyag <realevenyag@gmail.com>

* refactor: rename num_pk_columns to field_column_start

So we can support different format later

Signed-off-by: evenyag <realevenyag@gmail.com>

* chore: address comment

Signed-off-by: evenyag <realevenyag@gmail.com>

---------

Signed-off-by: evenyag <realevenyag@gmail.com>

...

This commit is contained in:

Yingwen

2025-08-14 22:38:47 +08:00

committed by GitHub

parent 021ad09c21

commit 60e01c7c3d

1 changed files with 920 additions and 12 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

932

src/mito2/src/read/flat_dedup.rs

File diff suppressed because it is too large Load Diff