e05c0cd87e
* Make `npm run docs` fail if there are any warnings. This will catch items missing from the API reference. * Add a check in our CI to make sure `npm run dos` runs without warnings and doesn't generate any new files (indicating it might be out-of-date. * Hide constructors that aren't user facing. * Remove unused enum `WriteMode`. Closes #2068
1.7 KiB
@lancedb/lancedb • Docs
@lancedb/lancedb / FtsOptions
Interface: FtsOptions
Options to create a full text search index
Properties
asciiFolding?
optional asciiFolding: boolean;
whether to remove punctuation
baseTokenizer?
optional baseTokenizer: "raw" | "simple" | "whitespace";
The tokenizer to use when building the index. The default is "simple".
The following tokenizers are available:
"simple" - Simple tokenizer. This tokenizer splits the text into tokens using whitespace and punctuation as a delimiter.
"whitespace" - Whitespace tokenizer. This tokenizer splits the text into tokens using whitespace as a delimiter.
"raw" - Raw tokenizer. This tokenizer does not split the text into tokens and indexes the entire text as a single token.
language?
optional language: string;
language for stemming and stop words
this is only used when stem or remove_stop_words is true
lowercase?
optional lowercase: boolean;
whether to lowercase tokens
maxTokenLength?
optional maxTokenLength: number;
maximum token length tokens longer than this length will be ignored
removeStopWords?
optional removeStopWords: boolean;
whether to remove stop words
stem?
optional stem: boolean;
whether to stem tokens
withPosition?
optional withPosition: boolean;
Whether to build the index with positions. True by default. If set to false, the index will not store the positions of the tokens in the text, which will make the index smaller and faster to build, but will not support phrase queries.