PSeitz 514d23a20c move tokenizer API to seperate crate (#1767) ...

closes #1766

Finding tantivy tokenizers is a frustrating experience currently, since
they need be updated for each tantivy version. That's unnecessary since
the API is rather stable anyway.

2023-01-09 06:37:38 +01:00

..

stop_word_filter

Make the built-in stop word lists selectable via the Language enum already used by the Stemmer filter. (#1671)

2022-11-15 17:40:25 +09:00

alphanum_only.rs

Add unit tests for tokenizers and filters (#1156)

2021-09-27 10:22:01 +09:00

ascii_folding_filter.rs

Removes all usage of block_on, and use a oneshot channel instead. (#1315)

2022-03-18 16:54:58 +09:00

empty_tokenizer.rs

Added JSON Type (#1270)

2022-02-24 16:25:22 +09:00

facet_tokenizer.rs

Dynamic analyzer (#755)

2020-01-29 18:23:37 +09:00

lower_caser.rs

Clippy comments (#1316)

2022-03-17 18:57:55 +09:00

mod.rs

move tokenizer API to seperate crate (#1767)

2023-01-09 06:37:38 +01:00

ngram_tokenizer.rs

move tokenizer API to seperate crate (#1767)

2023-01-09 06:37:38 +01:00

raw_tokenizer.rs

Add unit tests for tokenizers and filters (#1156)

2021-09-27 10:22:01 +09:00

remove_long.rs

Minor refactoring (#1266)

2022-01-28 15:55:55 +09:00

simple_tokenizer.rs

Enable usage of FuzzyTermQuery for specific fields via QueryParser (#1750)

2023-01-04 18:11:27 +09:00

split_compound_words.rs

Add dictionary-based SplitCompoundWords token filter.

2022-10-27 08:30:33 +02:00

stemmer.rs

Avoid unconditional allocation in StemmerTokenStream.

2022-10-26 18:11:15 +02:00

tokenized_string.rs

Improvements to doc linking, grammar, etc.

2022-09-19 18:10:22 +07:00

tokenizer_manager.rs

doc fix (#1391)

2022-06-21 15:53:33 +09:00

tokenizer.rs

move tokenizer API to seperate crate (#1767)

2023-01-09 06:37:38 +01:00

whitespace_tokenizer.rs

Enable usage of FuzzyTermQuery for specific fields via QueryParser (#1750)

2023-01-04 18:11:27 +09:00