mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-17 01:02:28 +00:00
Brendan Clement
758f738b8c
This PR is part of the documentation revamp. Once the full set of PRs is merged, we’ll publish the combined site to staging for review before updating docs.lancedb.com. Ingestion, search and integration examples are incomplete or inconsistent across languages, and some guidance misstates SDK behavior. This brings the guides and their example sources together. It includes complete Python, TypeScript and Rust workflows, isolated provider and reranker examples, the Superlinked corrections, and guidance on retries that can duplicate writes. Stacked on #4491. Each MDX file has one PR owner. Rebase onto main after #4491 is squash-merged. Local assembly, Mint validation and link checks pass. Example code is unchanged from the previously validated package. Publication stays held through integration.
59 lines
2.2 KiB
Plaintext
59 lines
2.2 KiB
Plaintext
---
|
|
title: "Polars"
|
|
sidebarTitle: "Polars & Arrow"
|
|
availability:
|
|
oss: available
|
|
enterprise: available
|
|
summary: >-
|
|
Polars data goes into `create_table`, and query results come back as Polars
|
|
DataFrames, for local and deployment tables alike; a LazyFrame over a whole
|
|
table is local only.
|
|
|
|
---
|
|
|
|
import {
|
|
PyPlatformsPolarsCreateTable,
|
|
PyPlatformsPolarsImports,
|
|
PyPlatformsPolarsLazyframe,
|
|
PyPlatformsPolarsPydantic,
|
|
PyPlatformsPolarsVectorSearch,
|
|
} from '/snippets/integrations.mdx';
|
|
|
|
LanceDB supports [Polars](https://github.com/pola-rs/polars), a blazingly fast DataFrame library for Python written in Rust. Under the hood, both Lance and Polars speak Arrow, so passing data back and forth stays zero-copy and ergonomic.
|
|
|
|
## Create and Query a Table {#create-and-query-a-table}
|
|
|
|
Import the required libraries, including the optional Pydantic helpers if you plan to define schemas.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsImports}
|
|
</CodeBlock>
|
|
|
|
Build a Polars `DataFrame`, convert it to Arrow, and use it directly when creating a LanceDB table.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsCreateTable}
|
|
</CodeBlock>
|
|
|
|
Run vector search and keep the results as a Polars `DataFrame` for further processing or visualization.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsVectorSearch}
|
|
</CodeBlock>
|
|
|
|
## Work with LazyFrames {#work-with-lazyframes}
|
|
|
|
When you want to operate on the entire table (potentially larger than RAM), convert to a Polars `LazyFrame` so you can chain transformations without loading everything at once. This works for local tables only; with LanceDB Enterprise, `search(...).to_polars()` returns an eager DataFrame of query results.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsLazyframe}
|
|
</CodeBlock>
|
|
|
|
## Define Schemas with Pydantic {#define-schemas-with-pydantic}
|
|
|
|
You can also describe your table via `LanceModel` and continue ingesting data from Polars. This is useful when multiple teams share a schema or when you want validation.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsPydantic}
|
|
</CodeBlock>
|