mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 23:32:27 +00:00
Jack Ye
b21fb2a4dc
The pages describing this code lived in lancedb/docs, a repository with ten stars that neither contributing guide mentioned. A user who hit a wrong page while reading the source had to find a separate, near-invisible repository to fix it. These 102 pages now sit beside the code, and a documentation fix is a pull request in the repository the reader already had open. `docs/web` is a complete Mintlify site: `cd docs/web && mint dev` serves it with no other checkout. Enterprise pages stay private and are merged in when the published site is assembled, so links into them do not resolve here — every open-source page does. `docs/web-tests` holds the tests the code examples are extracted from. They ran against a published wheel before; here they can run against this repository's own SDK, so an example that stops compiling fails the pull request that broke it rather than being noticed later. Both directories sit beside `docs/src` and are invisible to it: mkdocs reads `docs_dir: src`, so the SDK reference builds and deploys exactly as before.
52 lines
1.8 KiB
Plaintext
52 lines
1.8 KiB
Plaintext
---
|
|
title: "Polars"
|
|
sidebarTitle: "Polars & Arrow"
|
|
|
|
---
|
|
|
|
import {
|
|
PyPlatformsPolarsCreateTable,
|
|
PyPlatformsPolarsImports,
|
|
PyPlatformsPolarsLazyframe,
|
|
PyPlatformsPolarsPydantic,
|
|
PyPlatformsPolarsVectorSearch,
|
|
} from '/snippets/integrations.mdx';
|
|
|
|
LanceDB supports [Polars](https://github.com/pola-rs/polars), a blazingly fast DataFrame library for Python written in Rust. Under the hood, both Lance and Polars speak Arrow, so passing data back and forth stays zero-copy and ergonomic.
|
|
|
|
## Create and Query a Table {#create-and-query-a-table}
|
|
|
|
Import the required libraries, including the optional Pydantic helpers if you plan to define schemas.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsImports}
|
|
</CodeBlock>
|
|
|
|
Build a Polars `DataFrame`, convert it to Arrow, and use it directly when creating a LanceDB table.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsCreateTable}
|
|
</CodeBlock>
|
|
|
|
Run vector search and keep the results as a Polars `DataFrame` for further processing or visualization.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsVectorSearch}
|
|
</CodeBlock>
|
|
|
|
## Work with LazyFrames {#work-with-lazyframes}
|
|
|
|
When you want to operate on the entire table (potentially larger than RAM), convert to a Polars `LazyFrame` so you can chain transformations without loading everything at once.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsLazyframe}
|
|
</CodeBlock>
|
|
|
|
## Define Schemas with Pydantic {#define-schemas-with-pydantic}
|
|
|
|
You can also describe your table via `LanceModel` and continue ingesting data from Polars. This is useful when multiple teams share a schema or when you want validation.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsPydantic}
|
|
</CodeBlock>
|