docs: declare where storage, training and integration pages apply

Storage configuration from the connection is embedded only, since a
deployment configures its own. Training data loading, PyTorch, embedding
functions, pandas, Polars and Pydantic work with local and deployment
tables, and the Hugging Face Hub integration is embedded only.

A local table cannot be pickled for DataLoader workers, so the PyTorch
page says to wrap it in a Permutation; a LazyFrame over a whole table,
to_pandas in the LangChain example and to_arrow in the DuckDB note are
local only; and query text is embedded in the client for both offerings.
This commit is contained in:
Brendan Clement
2026-10-08 21:31:42 -07:00
parent ac2db48af6
commit 98d13f08c8
10 changed files with 61 additions and 5 deletions
+1 -1
View File
@@ -6,7 +6,7 @@ description: "Learn how to use the DuckDB-Lance extension to query Lance tables
LanceDB integrates with [DuckDB](https://duckdb.org/) through the [Lance extension](https://github.com/lance-format/lance-duckdb) for DuckDB. In this page, we'll show how LanceDB manages table lifecycle, and DuckDB provides SQL analytics (including joins) and search over those tables.
Note that earlier versions of LanceDB used to recommend converting Lance tables to Arrow tables via `table.to_arrow()`. Although this method is still available (because DuckDB [natively scans Arrow tables](https://duckdb.org/2021/12/03/duck-arrow)), it is no longer the recommended workflow for working with Lance tables in DuckDB. This page shows how to use the Lance extension with namespace-attached LanceDB tables, allowing you to pushdown SQL queries directly to the Lance layer.
Note that earlier versions of LanceDB used to recommend converting Lance tables to Arrow tables via `table.to_arrow()`. Although this method is still available for local tables (because DuckDB [natively scans Arrow tables](https://duckdb.org/2021/12/03/duck-arrow)), it is no longer the recommended workflow for working with Lance tables in DuckDB. This page shows how to use the Lance extension with namespace-attached LanceDB tables, allowing you to pushdown SQL queries directly to the Lance layer.
## Install {#install}
@@ -1,6 +1,12 @@
---
title: "Pandas and PyArrow"
sidebarTitle: "Pandas & PyArrow"
availability:
oss: available
enterprise: available
summary: >-
DataFrames go into `create_table` and come back from query results the same
way for local and deployment tables.
---
+8 -1
View File
@@ -1,6 +1,13 @@
---
title: "Polars"
sidebarTitle: "Polars & Arrow"
availability:
oss: available
enterprise: available
summary: >-
Polars data goes into `create_table`, and query results come back as Polars
DataFrames, for local and deployment tables alike; a LazyFrame over a whole
table is local only.
---
@@ -36,7 +43,7 @@ Run vector search and keep the results as a Polars `DataFrame` for further proce
## Work with LazyFrames {#work-with-lazyframes}
When you want to operate on the entire table (potentially larger than RAM), convert to a Polars `LazyFrame` so you can chain transformations without loading everything at once.
When you want to operate on the entire table (potentially larger than RAM), convert to a Polars `LazyFrame` so you can chain transformations without loading everything at once. This works for local tables only; against a deployment, `search(...).to_polars()` returns an eager DataFrame of query results.
<CodeBlock filename="Python" language="Python" icon="python">
{PyPlatformsPolarsLazyframe}
+7
View File
@@ -1,6 +1,13 @@
---
title: "Pydantic"
sidebarTitle: "Pydantic"
availability:
oss: available
enterprise: available
summary: >-
The SDK converts `LanceModel` schemas to Arrow, and query rows back to
models, in your process, so they work the same with local and deployment
tables.
---