mirror of
https://github.com/lancedb/lancedb.git
synced 2025-12-29 16:02:58 +00:00
f485378ea4
This is v1 of integrating full text search index into LanceDB.
# API
The query API is roughly the same as before, except if the input is text
instead of a vector we assume that its fts search.
## Example
If `table` is a LanceDB LanceTable, then:
Build index: `table.create_fts_index("text")`
Query: `df = table.search("puppy").limit(10).select(["text"]).to_df()`
# Implementation
Here we use the tantivy-py package to build the index. We then use the
row id's as the full-text-search index's doc id then we just do a Take
operation to fetch the rows.
# Limitations
1. don't support incremental row appends yet. New data won't show up in
search
2. local filesystem only
3. requires building tantivy explicitly
---------
Co-authored-by: Chang She <chang@lancedb.com>
33 lines
585 B
YAML
33 lines
585 B
YAML
site_name: LanceDB Documentation
|
|
docs_dir: src
|
|
|
|
theme:
|
|
name: "material"
|
|
features:
|
|
- content.code.copy
|
|
|
|
plugins:
|
|
- search
|
|
- mkdocstrings:
|
|
handlers:
|
|
python:
|
|
paths: [../python]
|
|
- mkdocs-jupyter
|
|
|
|
nav:
|
|
- Home: index.md
|
|
- Basics: basic.md
|
|
- Embeddings: embedding.md
|
|
- Indexing: ann_indexes.md
|
|
- Full-text search: fts.md
|
|
- Integrations: integrations.md
|
|
- Python API: python.md
|
|
|
|
markdown_extensions:
|
|
- pymdownx.highlight:
|
|
anchor_linenums: true
|
|
line_spans: __span
|
|
pygments_lang_class: true
|
|
- pymdownx.inlinehilite
|
|
- pymdownx.snippets
|
|
- pymdownx.superfences |