rust/tantivy
1
0
Fork 0
mirror of https://github.com/quickwit-oss/tantivy.git synced 2025-12-23 02:29:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

Updated columnar todo

Browse Source
This commit is contained in:
Paul Masurel
2023-03-21 18:55:23 +09:00
parent 8f7f1d6be4
commit ab703486aa
1 changed files with 11 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

33
columnar/src/TODO.md
View File

@@ -1,28 +1,22 @@
# zero to one # zero to one
* revisit line codec * revisit line codec
* add columns from schema on merge
* Plugging JSON
* replug examples
* move datetime to quickwit common
* switch to nanos * switch to nanos
* reintroduce the gcd map. * reintroduce the gcd map.
# Perf and Size # Perf and Size
* remove alloc in `ord_to_term`
+ multivaued range queries restrat frm the beginning all of the time. + multivaued range queries restrat frm the beginning all of the time.
* re-add ZSTD compression for dictionaries * re-add ZSTD compression for dictionaries
no systematic monotonic mapping * no systematic monotonic mapping
consider removing multilinear * consider removing multilinear
f32? * f32?
adhoc solution for bool? * adhoc solution for bool?
add metrics helper for aggregate. sum(row_id) * add metrics helper for aggregate. sum(row_id)
review inline absence/presence * review inline absence/presence
improv perf of select using PDEP * improv perf of select using PDEP
compare with roaring bitmap/elias fano etc etc. * compare with roaring bitmap/elias fano etc etc.
SIMD range? (see blog post) * SIMD range? (see blog post)
Add alignment? * Add alignment?
Consider another codec to bridge the gap between few and 5k elements * Consider another codec to bridge the gap between few and 5k elements
# Cleanup and rationalization # Cleanup and rationalization
in benchmark, unify percent vs ratio, f32 vs f64. in benchmark, unify percent vs ratio, f32 vs f64.
@@ -30,15 +24,10 @@ investigate if should have better errors? io::Error is overused at the moment.
rename rank/select in unit tests rename rank/select in unit tests
Review the public API via cargo doc Review the public API via cargo doc
go through TODOs go through TODOs
remove all doc_id occurences -> row_id
use the rank & select naming in unit tests branch.
multi-linear -> blockwise multi-linear -> blockwise
linear codec -> simply a multiplication for the index column linear codec -> simply a multiplication for the index column
rename columnar to something more explicit, like column_dictionary or columnar_table
rename fastfield -> column
document changes document changes
rationalization FastFieldValue, HasColumnType rationalization FastFieldValue, HasColumnType
isolate u128_based and uniform naming
# Other # Other
fix enhance column-cli fix enhance column-cli