mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-17 17:22:27 +00:00
docs: complete ingestion, search and integration examples (#4533)
This PR is part of the documentation revamp. Once the full set of PRs is merged, we’ll publish the combined site to staging for review before updating docs.lancedb.com. Ingestion, search and integration examples are incomplete or inconsistent across languages, and some guidance misstates SDK behavior. This brings the guides and their example sources together. It includes complete Python, TypeScript and Rust workflows, isolated provider and reranker examples, the Superlinked corrections, and guidance on retries that can duplicate writes. Stacked on #4491. Each MDX file has one PR owner. Rebase onto main after #4491 is squash-merged. Local assembly, Mint validation and link checks pass. Example code is unchanged from the previously validated package. Publication stays held through integration.
This commit is contained in:
@@ -6,7 +6,7 @@ description: "Learn how to use the DuckDB-Lance extension to query Lance tables
|
||||
|
||||
LanceDB integrates with [DuckDB](https://duckdb.org/) through the [Lance extension](https://github.com/lance-format/lance-duckdb) for DuckDB. In this page, we'll show how LanceDB manages table lifecycle, and DuckDB provides SQL analytics (including joins) and search over those tables.
|
||||
|
||||
Note that earlier versions of LanceDB used to recommend converting Lance tables to Arrow tables via `table.to_arrow()`. Although this method is still available (because DuckDB [natively scans Arrow tables](https://duckdb.org/2021/12/03/duck-arrow)), it is no longer the recommended workflow for working with Lance tables in DuckDB. This page shows how to use the Lance extension with namespace-attached LanceDB tables, allowing you to pushdown SQL queries directly to the Lance layer.
|
||||
Note that earlier versions of LanceDB used to recommend converting Lance tables to Arrow tables via `table.to_arrow()`. Although this method is still available for local tables (because DuckDB [natively scans Arrow tables](https://duckdb.org/2021/12/03/duck-arrow)), it is no longer the recommended workflow for working with Lance tables in DuckDB. This page shows how to use the Lance extension with namespace-attached LanceDB tables, allowing you to pushdown SQL queries directly to the Lance layer.
|
||||
|
||||
|
||||
## Install {#install}
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
---
|
||||
title: "Pandas and PyArrow"
|
||||
sidebarTitle: "Pandas & PyArrow"
|
||||
availability:
|
||||
oss: available
|
||||
enterprise: available
|
||||
summary: >-
|
||||
DataFrames go into `create_table` and come back from query results the same
|
||||
way for local and deployment tables.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
---
|
||||
title: "Polars"
|
||||
sidebarTitle: "Polars & Arrow"
|
||||
availability:
|
||||
oss: available
|
||||
enterprise: available
|
||||
summary: >-
|
||||
Polars data goes into `create_table`, and query results come back as Polars
|
||||
DataFrames, for local and deployment tables alike; a LazyFrame over a whole
|
||||
table is local only.
|
||||
|
||||
---
|
||||
|
||||
@@ -36,7 +43,7 @@ Run vector search and keep the results as a Polars `DataFrame` for further proce
|
||||
|
||||
## Work with LazyFrames {#work-with-lazyframes}
|
||||
|
||||
When you want to operate on the entire table (potentially larger than RAM), convert to a Polars `LazyFrame` so you can chain transformations without loading everything at once.
|
||||
When you want to operate on the entire table (potentially larger than RAM), convert to a Polars `LazyFrame` so you can chain transformations without loading everything at once. This works for local tables only; with LanceDB Enterprise, `search(...).to_polars()` returns an eager DataFrame of query results.
|
||||
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{PyPlatformsPolarsLazyframe}
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
---
|
||||
title: "Pydantic"
|
||||
sidebarTitle: "Pydantic"
|
||||
availability:
|
||||
oss: available
|
||||
enterprise: available
|
||||
summary: >-
|
||||
The SDK converts `LanceModel` schemas to Arrow, and query rows back to
|
||||
models, in your process, so they work the same with local and deployment
|
||||
tables.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user