mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-30 15:26:52 +00:00
Will JonesandClaude Opus 5.5
4d7a64712f
## Summary Adopts the product terminology guide across the docs site pages, SDK docstrings and error messages. "LanceDB OSS" and local mode become **LanceDB Embedded**. "Remote" as the execution model becomes **LanceDB Distributed**, which LanceDB Enterprise provides. Where free vs paid is the point, pages contrast "LanceDB Embedded (open source)" with "LanceDB Enterprise". This is a first trial of the Embedded name, so reviewers can judge it in context. - **Embedded vs Distributed** for where execution runs (`optimize()`, consistency, branching, maintenance). Main pages define it once: "LanceDB Enterprise runs LanceDB Distributed: …" - **LanceDB Cloud** is kept only for legacy-Cloud specifics (`db://` region, the brute-force KNN limit) and marked legacy. "Cloud and Enterprise" becomes Enterprise or Distributed. - **Plan executor** becomes "executor" or the fleet name. - **Error messages** change, e.g. `optimize is not supported for remote tables.` → `... in LanceDB Distributed.` Tests that match on them are updated. - **Vale rules** (`EmbeddedName`, `OSSProduct`, `RemoteExecution`, `DistributedEnterprise`, `LegacyCloud`, `InternalNames`) enforce the terms in prose. Unchanged: identifiers (`availability.oss`, the `remote` module and feature, `RemoteTable`, `db://`), heading anchors, internal code comments, and Geneva. ## Evidence - Rust lib tests for the changed messages and the Python tests in `test_remote_db.py`, `test_materialized_views.py` and `test_table.py` pass locally. - Not run locally: - The Node tests: `remote.test.ts` and `table.test.ts` have updated assertions. - Vale. - The `experimental_functions`-marked materialized-view tests. - `docs/src` API reference was hand-edited to match the docstrings rather than regenerated. ## Merge Risk **Door:** two-way; prose and message text only. **Blast Radius:** `docs`, `error-messages`: callers matching on old error strings would break. Companion PRs: lancedb/docs (assembler labels, STYLE.md) and sophon (Enterprise pages). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
59 lines
2.2 KiB
Plaintext
59 lines
2.2 KiB
Plaintext
---
|
|
title: "Polars"
|
|
sidebarTitle: "Polars & Arrow"
|
|
availability:
|
|
oss: available
|
|
enterprise: available
|
|
summary: >-
|
|
Polars data goes into `create_table`, and query results come back as Polars
|
|
DataFrames, for local and deployment tables alike; a LazyFrame over a whole
|
|
table is local only.
|
|
|
|
---
|
|
|
|
import {
|
|
PyPlatformsPolarsCreateTable,
|
|
PyPlatformsPolarsImports,
|
|
PyPlatformsPolarsLazyframe,
|
|
PyPlatformsPolarsPydantic,
|
|
PyPlatformsPolarsVectorSearch,
|
|
} from '/snippets/integrations.mdx';
|
|
|
|
LanceDB supports [Polars](https://github.com/pola-rs/polars), a blazingly fast DataFrame library for Python written in Rust. Under the hood, both Lance and Polars speak Arrow, so passing data back and forth stays zero-copy and ergonomic.
|
|
|
|
## Create and Query a Table {#create-and-query-a-table}
|
|
|
|
Import the required libraries, including the optional Pydantic helpers if you plan to define schemas.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsImports}
|
|
</CodeBlock>
|
|
|
|
Build a Polars `DataFrame`, convert it to Arrow, and use it directly when creating a LanceDB table.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsCreateTable}
|
|
</CodeBlock>
|
|
|
|
Run vector search and keep the results as a Polars `DataFrame` for further processing or visualization.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsVectorSearch}
|
|
</CodeBlock>
|
|
|
|
## Work with LazyFrames {#work-with-lazyframes}
|
|
|
|
When you want to operate on the entire table (potentially larger than RAM), convert to a Polars `LazyFrame` so you can chain transformations without loading everything at once. This works for LanceDB Embedded tables only; with LanceDB Enterprise, `search(...).to_polars()` returns an eager DataFrame of query results.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsLazyframe}
|
|
</CodeBlock>
|
|
|
|
## Define Schemas with Pydantic {#define-schemas-with-pydantic}
|
|
|
|
You can also describe your table via `LanceModel` and continue ingesting data from Polars. This is useful when multiple teams share a schema or when you want validation.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyPlatformsPolarsPydantic}
|
|
</CodeBlock>
|