mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-16 08:42:29 +00:00
This PR is part of the documentation revamp. Once the full set of PRs is merged, we’ll publish the combined site to staging for review before updating docs.lancedb.com. The curation, SQL and operations pages contain outdated claims and examples that do not reliably work as written. This corrects those pages against the reviewed SDK and local server evidence, including prepared statements, background jobs, cleanup and monitoring. Integration guides are grouped with their examples in #4533. Function and Secret-object documentation stays unpublished until its release. The two Function pages are listed in `.mintignore` and left out of the navigation, and the sections about them on other pages are MDX comments. Stacked on #4533. Each MDX file has one PR owner. Rebase onto main after #4533 is squash-merged. Local assembly, Mint validation and link checks pass, and the assembled site shows no Function or Secret-object content. A local release rehearsal that uncomments the drafts and restores the navigation also passes. Hold publication until this PR and lancedb/sophon#8688 are on `main`.
149 lines
4.2 KiB
Plaintext
149 lines
4.2 KiB
Plaintext
---
|
|
title: "Full-Text Search"
|
|
sidebarTitle: "Full-Text Search"
|
|
description: "The fts() table function — match, fuzzy, phrase, boolean, boost and multi-match queries from SQL."
|
|
icon: magnifying-glass
|
|
keywords: ["fts", "sql", "full text search", "fuzzy", "phrase", "boolean", "boost", "multi-match"]
|
|
---
|
|
|
|
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
|
|
|
|
Full-text search is a **table function**, `fts(table, query)`, rather than a
|
|
predicate -- it produces a relevance score alongside the matching rows, and a
|
|
`WHERE` clause has nowhere to put one. The two string-literal arguments are
|
|
the table name and a JSON-encoded query, built with the same query-builder
|
|
classes used from the SDK and serialized with `.to_json()`.
|
|
|
|
## Setup {#setup}
|
|
|
|
Run these statements in order, sending one statement per SQL request.
|
|
Semicolons separate the statements for clients that execute a script.
|
|
|
|
```sql
|
|
CREATE TABLE my_docs (id INT, text STRING, category STRING, author STRING);
|
|
|
|
INSERT INTO my_docs VALUES
|
|
(1, 'The happy puppy runs merrily in the park', 'animals', 'Alice'),
|
|
(2, 'A curious kitten jumps quickly over the fence', 'animals', 'Bob'),
|
|
(3, 'The puppy catches a ball with great enthusiasm', 'sports', 'Alice'),
|
|
(5, 'Puppy training requires patience and dedication', 'training', 'Alice');
|
|
|
|
CREATE INDEX ON my_docs USING fts (text);
|
|
```
|
|
|
|
<Warning>
|
|
Phrase queries need position information, which costs extra index size --
|
|
add `WITH (with_position = true)` to `CREATE INDEX`. Without it,
|
|
[phrase queries](#phrase) find nothing. See
|
|
[FTS parameters](/indexing/fts-index#fts-parameters).
|
|
</Warning>
|
|
|
|
## Basic search {#basic}
|
|
|
|
The query builder produces the JSON string used as the second SQL argument.
|
|
The SQL examples below spell out the equivalent match object so they can be
|
|
run directly after the setup above.
|
|
|
|
```python
|
|
from lancedb.query import MatchQuery
|
|
|
|
query = MatchQuery("puppy", "text")
|
|
json_query = query.to_json()
|
|
```
|
|
|
|
```sql
|
|
SELECT id, text, category
|
|
FROM fts('my_docs', '{"match":{"column":"text","terms":"puppy"}}')
|
|
LIMIT 5
|
|
```
|
|
|
|
By default the top 5 matches come back in **arbitrary order** -- select
|
|
`_score` and order by it for relevance ranking:
|
|
|
|
```sql
|
|
SELECT id, text, _score
|
|
FROM fts('my_docs', '{"match":{"column":"text","terms":"puppy"}}')
|
|
ORDER BY _score DESC
|
|
LIMIT 5
|
|
```
|
|
|
|
`_score` is a pseudo-column carrying the query's BM25 relevance for the row;
|
|
it behaves like any other column, but exists only in a relation that came
|
|
from `fts()`. See [Data types](/sql/data-types#pseudo-columns).
|
|
|
|
## Fuzzy {#fuzzy}
|
|
|
|
Tolerates typos, up to a character-edit budget:
|
|
|
|
```python
|
|
MatchQuery("pupy", "text", fuzziness=2)
|
|
```
|
|
|
|
## Phrase {#phrase}
|
|
|
|
Exact phrase, or with `slop` to allow words between terms:
|
|
|
|
```python
|
|
from lancedb.query import PhraseQuery
|
|
|
|
PhraseQuery("happy puppy", "text")
|
|
PhraseQuery("puppy park", "text", slop=2) # up to 2 words between them
|
|
```
|
|
|
|
## Boolean {#boolean}
|
|
|
|
Combine queries with `&` (AND) or `|` (OR):
|
|
|
|
```python
|
|
MatchQuery("puppy", "text") & MatchQuery("happy", "text")
|
|
MatchQuery("puppy", "text") | MatchQuery("kitten", "text")
|
|
```
|
|
|
|
## Boost {#boost}
|
|
|
|
Weight a query up or down relative to another:
|
|
|
|
```python
|
|
from lancedb.query import BoostQuery
|
|
|
|
BoostQuery(
|
|
positive=MatchQuery("puppy", "text"),
|
|
negative=MatchQuery("kitten", "text"),
|
|
negative_boost=0.2,
|
|
)
|
|
```
|
|
|
|
## Multi-match {#multi-match}
|
|
|
|
Search several columns at once, optionally weighting them differently:
|
|
|
|
```python
|
|
from lancedb.query import MultiMatchQuery
|
|
|
|
MultiMatchQuery("puppy", ["text", "category"])
|
|
MultiMatchQuery("puppy", ["text", "category"], boosts=[2.0, 1.0]) # text weighted 2x
|
|
```
|
|
|
|
## Combining with SQL {#combining}
|
|
|
|
`fts()` is a relation, so it composes with the rest of the SQL surface --
|
|
filter, join, or aggregate over it same as any table:
|
|
|
|
```sql
|
|
SELECT id, text, category
|
|
FROM fts('my_docs', '{"match":{"column":"text","terms":"puppy"}}')
|
|
WHERE category = 'animals'
|
|
LIMIT 5
|
|
```
|
|
|
|
## What to read next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Full-Text Search" icon="magnifying-glass" href="/search/full-text-search">
|
|
The query-builder equivalent, and every query form in full.
|
|
</Card>
|
|
<Card title="FTS Index" icon="bolt" href="/indexing/fts-index">
|
|
Tokenizers, and the position information phrase queries need.
|
|
</Card>
|
|
</CardGroup>
|