Files
lancedb/docs/web/sql/dql/fts.mdx
T
Jack Ye dd449978d9 docs: clarify Function workflows and search contracts
Clarify Function signatures, refresh and recovery workflows, job snapshots, FTS operators, and IVF deduplication limits.
2026-11-14 01:09:40 -08:00

236 lines
6.6 KiB
Plaintext

---
title: "Full-Text Search"
sidebarTitle: "Full-Text Search"
description: "The fts() table function: match, fuzzy, phrase, boolean, boost and multi-match queries from SQL."
icon: magnifying-glass
keywords: ["fts", "sql", "full text search", "fuzzy", "phrase", "boolean", "boost", "multi-match"]
availability:
embedded: unavailable
enterprise: available
---
Full-text search is a **table function**, `fts(table, query)`, rather than a
predicate: it produces a relevance score alongside the matching rows, and a
`WHERE` clause has nowhere to put one. The two string-literal arguments are
the table name and a JSON-encoded query, built with the same query-builder
classes used from the SDK and serialized with `.to_json()`.
## Setup {#setup}
Run these statements in order, sending one statement per SQL request.
Semicolons separate the statements for clients that execute a script.
<CodeGroup>
```sql SQL icon="database"
CREATE TABLE my_docs (id INT, text STRING, category STRING, author STRING);
INSERT INTO my_docs VALUES
(1, 'The happy puppy runs merrily in the park', 'animals', 'Alice'),
(2, 'A curious kitten jumps quickly over the fence', 'animals', 'Bob'),
(3, 'The puppy catches a ball with great enthusiasm', 'sports', 'Alice'),
(5, 'Puppy training requires patience and dedication', 'training', 'Alice');
CREATE INDEX ON my_docs USING fts (text) WITH (with_position = true);
```
</CodeGroup>
<Warning>
Phrase queries need position information, which costs extra index size.
The setup above enables it with `WITH (with_position = true)`. An existing
index without positions must be rebuilt before running a phrase query; the
server rejects the query and asks you to recreate the index with positions.
It does not silently return no matches. See
[FTS parameters](/indexing/fts-index#fts-parameters).
</Warning>
## Basic search {#basic}
The query builder produces the JSON string used as the second SQL argument.
The SQL examples below spell out the equivalent match object so they can be
run directly after the setup above.
<CodeGroup>
```python Python icon="python"
from lancedb.query import MatchQuery
query = MatchQuery("puppy", "text")
json_query = query.to_json()
```
</CodeGroup>
<CodeGroup>
```sql SQL icon="database"
SELECT id, text, category
FROM fts(
'my_docs',
'{"match":{"column":"text","terms":"puppy"}}'
)
LIMIT 5
```
</CodeGroup>
By default the top 5 matches come back in **arbitrary order**. Select
`_score` and order by it for relevance ranking:
<CodeGroup>
```sql SQL icon="database"
SELECT id, text, _score
FROM fts(
'my_docs',
'{"match":{"column":"text","terms":"puppy"}}'
)
ORDER BY _score DESC
LIMIT 5
```
</CodeGroup>
`_score` is a pseudo-column carrying the query's BM25 relevance for the row;
it behaves like any other column, but exists only in a relation that came
from `fts()`. See [Data types](/sql/data-types#pseudo-columns).
## Match any term or every term {#term-operators}
The default match operator is OR. `"happy puppy"` matches either analyzed
term. SQL JSON uses `"Or"` and `"And"`, with this casing; Python uses the
`FullTextOperator.OR` and `FullTextOperator.AND` enum members:
```python
from lancedb.query import FullTextOperator, MatchQuery
query = MatchQuery("happy puppy", "text", operator=FullTextOperator.AND)
json_query = query.to_json()
```
```sql
SELECT id, text, _score
FROM fts('my_docs', '{"match":{"column":"text","terms":"happy puppy","operator":"And"}}')
ORDER BY _score DESC
```
Both terms must occur, but they need not be adjacent. Do not embed `AND` or
`OR` in `terms` to select an operator.
## Count matches {#count}
Count the `fts()` relation before applying a top-k limit:
```sql
SELECT COUNT(*) AS matching_rows
FROM fts('my_docs', '{"match":{"column":"text","terms":"happy puppy","operator":"And"}}')
```
This is an Enterprise SQL workaround when the SDK's FTS query builder does
not expose a match-count operation. `table.count_rows()` without an FTS
relation counts table rows, and taking the length of a limited search counts
only that result page. For list-element indexes, multiple matching elements
can produce multiple rows; use `COUNT(DISTINCT id)` when you need unique
application records.
## Fuzzy {#fuzzy}
Tolerates typos, up to a character-edit budget:
<CodeGroup>
```python Python icon="python"
MatchQuery("pupy", "text", fuzziness=2)
```
</CodeGroup>
## Phrase {#phrase}
Exact phrase, or with `slop` to allow words between terms:
<CodeGroup>
```python Python icon="python"
from lancedb.query import PhraseQuery
PhraseQuery("happy puppy", "text")
PhraseQuery("puppy park", "text", slop=2) # up to 2 words between them
```
</CodeGroup>
```sql
SELECT id, text, _score
FROM fts('my_docs', '{"phrase":{"column":"text","terms":"happy puppy","slop":0}}')
ORDER BY _score DESC
```
## Boolean {#boolean}
Combine queries with `&` (AND) or `|` (OR):
<CodeGroup>
```python Python icon="python"
MatchQuery("puppy", "text") & MatchQuery("happy", "text")
MatchQuery("puppy", "text") | MatchQuery("kitten", "text")
```
</CodeGroup>
For SQL, serialize the composed query rather than writing occurrence enums
by hand:
```python
query = MatchQuery("puppy", "text") & MatchQuery("happy", "text")
json_query = query.to_json()
```
Use that string as the second `fts()` argument. `.to_json()` supplies the
serialized Boolean field names for the SDK version you use.
## Boost {#boost}
Weight a query up or down relative to another:
<CodeGroup>
```python Python icon="python"
from lancedb.query import BoostQuery
BoostQuery(
positive=MatchQuery("puppy", "text"),
negative=MatchQuery("kitten", "text"),
negative_boost=0.2,
)
```
</CodeGroup>
## Multi-match {#multi-match}
Search several columns at once, optionally weighting them differently:
<CodeGroup>
```python Python icon="python"
from lancedb.query import MultiMatchQuery
MultiMatchQuery("puppy", ["text", "category"])
MultiMatchQuery("puppy", ["text", "category"], boosts=[2.0, 1.0]) # text weighted 2x
```
</CodeGroup>
## Combining with SQL {#combining}
`fts()` is a relation, so it composes with the rest of the SQL surface.
Filter, join, or aggregate over it same as any table:
<CodeGroup>
```sql SQL icon="database"
SELECT id, text, category
FROM fts(
'my_docs',
'{"match":{"column":"text","terms":"puppy"}}'
)
WHERE category = 'animals'
LIMIT 5
```
</CodeGroup>
## What to read next {#next}
<CardGroup cols={2}>
<Card title="Full-Text Search" icon="magnifying-glass" href="/search/full-text-search">
The query-builder equivalent, and every query form in full.
</Card>
<Card title="FTS Index" icon="bolt" href="/indexing/fts-index">
Tokenizers, and the position information phrase queries need.
</Card>
</CardGroup>