Adam Reichold c32ab66bbd Small improvements to StopWorldFilter (#1657) ...

* Do not copy the whole set of stop words for each stream

* Make construction of StopWordFilter more flexible.

2022-11-01 16:47:34 +09:00

..

alphanum_only.rs

Add unit tests for tokenizers and filters (#1156)

2021-09-27 10:22:01 +09:00

ascii_folding_filter.rs

Removes all usage of block_on, and use a oneshot channel instead. (#1315)

2022-03-18 16:54:58 +09:00

empty_tokenizer.rs

Added JSON Type (#1270)

2022-02-24 16:25:22 +09:00

facet_tokenizer.rs

Dynamic analyzer (#755)

2020-01-29 18:23:37 +09:00

lower_caser.rs

Clippy comments (#1316)

2022-03-17 18:57:55 +09:00

mod.rs

Add dictionary-based SplitCompoundWords token filter.

2022-10-27 08:30:33 +02:00

ngram_tokenizer.rs

Fix warnings when doc'ing private items. (#1579)

2022-10-03 14:24:00 +09:00

raw_tokenizer.rs

Add unit tests for tokenizers and filters (#1156)

2021-09-27 10:22:01 +09:00

remove_long.rs

Minor refactoring (#1266)

2022-01-28 15:55:55 +09:00

simple_tokenizer.rs

Minor refactoring (#1266)

2022-01-28 15:55:55 +09:00

split_compound_words.rs

Add dictionary-based SplitCompoundWords token filter.

2022-10-27 08:30:33 +02:00

stemmer.rs

Avoid unconditional allocation in StemmerTokenStream.

2022-10-26 18:11:15 +02:00

stop_word_filter.rs

Small improvements to StopWorldFilter (#1657)

2022-11-01 16:47:34 +09:00

tokenized_string.rs

Improvements to doc linking, grammar, etc.

2022-09-19 18:10:22 +07:00

tokenizer_manager.rs

doc fix (#1391)

2022-06-21 15:53:33 +09:00

tokenizer.rs

Improvements to doc linking, grammar, etc.

2022-09-19 18:10:22 +07:00

whitespace_tokenizer.rs

Minor refactoring (#1266)

2022-01-28 15:55:55 +09:00