PSeitz 7b31100208 refactor vint (#2010) ...

- improve performance of vint
vint serialization shows up in performance profiles during indexing.
It would also make sense to limit the value space to u29 and operate on 4 bytes only.
- remove unused code
- add missing inlines
- fix regex test

2023-04-25 08:49:36 +02:00

..

stop_word_filter

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

alphanum_only.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

ascii_folding_filter.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

empty_tokenizer.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

facet_tokenizer.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

lower_caser.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

mod.rs

Expose to use it in quickwit.

2023-04-13 18:28:53 +02:00

ngram_tokenizer.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

raw_tokenizer.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

regex_tokenizer.rs

refactor vint (#2010)

2023-04-25 08:49:36 +02:00

remove_long.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

simple_tokenizer.rs

Making more things public for quickwit. (#2005)

2023-04-20 11:37:45 +09:00

split_compound_words.rs

Bump aho-corasick dependency to version 1.0 and adjust to API changes (#2002)

2023-04-18 07:34:30 +02:00

stemmer.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

tokenized_string.rs

Improvements to doc linking, grammar, etc.

2022-09-19 18:10:22 +07:00

tokenizer_manager.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

tokenizer.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00

whitespace_tokenizer.rs

refactor tokenization pipeline to use GATs (#1924)

2023-03-09 09:39:37 +01:00