rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-05-22 22:40:41 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(python): support model-backed native FTS tokenizers (#3289)

Browse Source 
This wires Lance's existing `jieba/*` and `lindera/*` native FTS
tokenizers through the Python SDK instead of leaving them behind
disabled features and narrow public typing. It also documents the
`LANCE_LANGUAGE_MODEL_HOME` model layout and adds Python coverage for
successful CJK indexing plus missing-model error guidance.

Closes #2168.
This commit is contained in:
Xuanwo
2026-05-08 23:53:14 +08:00
committed by GitHub
parent 694aa48e19
commit 9b21c136c6
12 changed files with 725 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
docs/src/python/python.md
View File

@@ -94,11 +94,11 @@ of raw SQL strings with [where][lancedb.query.LanceQueryBuilder.where] and
## Full text search
::: lancedb.fts.create_index
Use [lancedb.table.Table.create_fts_index][] for the synchronous API or
[lancedb.table.AsyncTable.create_index][] with [lancedb.index.FTS][] for the
asynchronous API.
::: lancedb.fts.populate_index
::: lancedb.fts.search_index
::: lancedb.index.FTS
## Utilities