Commit Graph
23 Commits
Author SHA1 Message Date
Paul Masurel d30cafa80a Fixes the behavior of the range doc set, especially on multivalued
cardinality.

We stop scanning from the beginning of the column which is a great
improvement.
This is done by introducing a stateful `SelectCursor`.
2023-02-13 04:02:14 +01:00
Paul Masurel b7bfa20e38 Fixed test performance. 2023-02-09 17:39:55 +01:00
Paul Masurel db8583db75 Fixing unit test 2023-02-09 16:53:05 +01:00
Paul Masurel 405e2cf4d9 Merge with main 2023-02-09 14:28:57 +01:00
Paul Masurel b63c6c27bc adding change from main 2023-02-09 14:18:46 +01:00
Paul Masurel bd5eea9852 Integrated columnar work. 2023-02-09 13:14:31 +01:00
Paul MasurelandPSeitz 2874554ee4 Removed the sorting logic that forced column type to be sorted like (#1816)
* Removed the sorting logic that forced column type to be sorted like
ColumnTypes.

* add comments

Co-authored-by: PSeitz <PSeitz@users.noreply.github.com>
2023-01-20 12:43:28 +01:00
PSeitz 226d0f88bc add columnar to workspace (#1808) 2023-01-20 11:47:10 +01:00
Paul Masurel 9548570e88 Fixing broken test build 2023-01-20 18:18:32 +09:00
Paul Masurel 9a296b29b7 Renamed dense file to dense.rs 2023-01-20 17:22:25 +09:00
PSeitzandPaul Masurel b31fd389d8 collect columns for merge (#1812)
* collect columns for merge

* return column_type from, fix visibility

* fix

Co-authored-by: Paul Masurel <paul@quickwit.io>
2023-01-20 07:58:29 +01:00
Paul Masurel 89cec79813 Make it possible to force a column type and intricate bugfix. (#1815) 2023-01-20 14:30:56 +09:00
Paul Masurel 08919a2900 Improvement on the scalar / random bitpacker code. (#1781)
* Improvement on the scalar / random bitpacker code.

Added proptesting
Added simple benchmark
Added assert and comments on the very non trivial hidden contract
Remove the need for an extra padding.

The last point introduces a small performance regression (~10%).

* Fixing unit tests
2023-01-19 18:09:13 +09:00
Paul Masurel e3d504d833 Minor code cleanup (#1810) 2023-01-19 17:47:26 +09:00
Paul Masurel 5a42c5aae9 Add support for multivalues (#1809) 2023-01-19 16:55:01 +09:00
Paul Masurel a86b104a40 Differentiating between str and bytes, + unit test 2023-01-19 14:38:12 +09:00
PSeitz f9abd256b7 add ip addr to columnar (#1805) 2023-01-19 05:36:06 +01:00
Paul Masurel 9f42b6440a Completed unit test for dictionary encoded column 2023-01-19 12:15:27 +09:00
Paul Masurel c723ed3f0b Columnar merge (#1806) 2023-01-19 11:52:27 +09:00
PSeitz c4af63e588 add rename (#1797) 2023-01-18 13:28:37 +09:00
Paul Masurel 25bad784ad Integrated fastfield codecs into columnar. (#1782)
Introduced asymetric OptionalCodec / SerializableOptionalCodec
Removed cardinality from the columnar sstable.
Added DynamicColumn
Reorganized all files
Change DenseCodec serialization logic.
Renamed methods to rank/select
Moved versioning footer to the columnar level
2023-01-16 17:24:49 +09:00
Paul Masurel 7a8fce0ae7 Minor mini fixes 2023-01-10 14:15:30 +09:00
Paul MasurelandPSeitz 4f9efe654c Support for columnar (#1734)
* Added support for dynamic fast field.

See README for more information.

* Apply suggestions from code review

Co-authored-by: PSeitz <PSeitz@users.noreply.github.com>
2023-01-07 17:37:00 +09:00