mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 07:12:27 +00:00
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).
This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.
Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.
**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.
**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.
Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.
**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.
The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.
---------
Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
120 lines
3.8 KiB
Plaintext
120 lines
3.8 KiB
Plaintext
---
|
|
title: "DuckDB"
|
|
sidebarTitle: "DuckDB"
|
|
description: "Learn how to use the DuckDB-Lance extension to query Lance tables with SQL."
|
|
---
|
|
|
|
LanceDB integrates with [DuckDB](https://duckdb.org/) through the [Lance extension](https://github.com/lance-format/lance-duckdb) for DuckDB. In this page, we'll show how LanceDB manages table lifecycle, and DuckDB provides SQL analytics (including joins) and search over those tables.
|
|
|
|
Note that earlier versions of LanceDB used to recommend converting Lance tables to Arrow tables via `table.to_arrow()`. Although this method is still available (because DuckDB [natively scans Arrow tables](https://duckdb.org/2021/12/03/duck-arrow)), it is no longer the recommended workflow for working with Lance tables in DuckDB. This page shows how to use the Lance extension with namespace-attached LanceDB tables, allowing you to pushdown SQL queries directly to the Lance layer.
|
|
|
|
|
|
## Install {#install}
|
|
|
|
Install the DuckDB CLI as per [their docs](https://duckdb.org/install) and alternatively, their Python package with `pip install duckdb`.
|
|
|
|
Then, open the DuckDB CLI and install and load the Lance extension as follows:
|
|
|
|
```sql SQL icon="database"
|
|
INSTALL lance;
|
|
LOAD lance;
|
|
```
|
|
|
|
## Attach the directory namespace in DuckDB {#attach-the-directory-namespace-in-duckdb}
|
|
|
|
Attach the LanceDB root directory as a Lance namespace:
|
|
|
|
```sql SQL icon="database"
|
|
ATTACH './local_lancedb' AS lance_ns (TYPE LANCE);
|
|
```
|
|
|
|
In this page, tables are referenced using `lance_ns.main.<table_name>`, so the table path is `lance_ns.main.lance_duck`.
|
|
|
|
## Write Lance table {#write-lance-table}
|
|
|
|
Create the `lance_duck` table using SQL and populate it with sample data:
|
|
|
|
```sql SQL icon="database"
|
|
CREATE OR REPLACE TABLE lance_ns.main.lance_duck AS
|
|
SELECT *
|
|
FROM (
|
|
VALUES
|
|
('duck', 'quack', [0.9, 0.7, 0.1]::FLOAT[]),
|
|
('horse', 'neigh', [0.3, 0.1, 0.5]::FLOAT[]),
|
|
('dragon', 'roar', [0.5, 0.2, 0.7]::FLOAT[])
|
|
) AS t(animal, noise, vector);
|
|
```
|
|
|
|
This table is the source of truth for all DuckDB queries below.
|
|
|
|
<Info>
|
|
The examples below show SQL entered in the DuckDB CLI. You can run the same SQL from
|
|
Python as well, using LanceDB and DuckDB's Python clients in your application code.
|
|
</Info>
|
|
|
|
## Query the table with SQL {#query-the-table-with-sql}
|
|
|
|
```sql SQL icon="database"
|
|
SELECT *
|
|
FROM lance_ns.main.lance_duck
|
|
LIMIT 5;
|
|
```
|
|
|
|
## Vector search {#vector-search}
|
|
|
|
```sql SQL icon="database"
|
|
SELECT animal, noise, vector, _distance
|
|
FROM lance_vector_search(
|
|
'lance_ns.main.lance_duck',
|
|
'vector',
|
|
[0.8, 0.7, 0.2]::FLOAT[],
|
|
k = 1,
|
|
prefilter = true
|
|
)
|
|
ORDER BY _distance ASC;
|
|
```
|
|
|
|
## Full-text search {#full-text-search}
|
|
|
|
```sql SQL icon="database"
|
|
SELECT animal, noise, vector, _score
|
|
FROM lance_fts(
|
|
'lance_ns.main.lance_duck',
|
|
'animal',
|
|
'the brave knight faced the dragon',
|
|
k = 1,
|
|
prefilter = true
|
|
)
|
|
ORDER BY _score DESC;
|
|
```
|
|
|
|
## Hybrid search {#hybrid-search}
|
|
|
|
```sql SQL icon="database"
|
|
SELECT animal, noise, vector, _hybrid_score, _distance, _score
|
|
FROM lance_hybrid_search(
|
|
'lance_ns.main.lance_duck',
|
|
'vector',
|
|
[0.8, 0.7, 0.2]::FLOAT[],
|
|
'animal',
|
|
'the duck surprised the dragon',
|
|
k = 2,
|
|
prefilter = false,
|
|
alpha = 0.5,
|
|
oversample_factor = 4
|
|
)
|
|
ORDER BY _hybrid_score DESC;
|
|
```
|
|
|
|
## Directory namespace model {#directory-namespace-model}
|
|
|
|
A directory namespace maps a LanceDB catalog root to namespace-qualified table identifiers in DuckDB. This keeps table discovery and table naming stable as your project grows.
|
|
|
|
To learn more about the catalog and namespace model, see [Namespaces and the Catalog Model](/management/namespace).
|
|
|
|
|
|
## Advanced usage {#advanced-usage}
|
|
|
|
See the [docs](https://github.com/lance-format/lance-duckdb) directory in the Lance-DuckDB extension repo
|
|
for more advanced usage on SQL and REST API clients.
|