mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 15:22:30 +00:00
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).
This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.
Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.
**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.
**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.
Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.
**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.
The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.
---------
Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
90 lines
3.3 KiB
Plaintext
90 lines
3.3 KiB
Plaintext
---
|
|
title: "Pydantic"
|
|
sidebarTitle: "Pydantic"
|
|
|
|
---
|
|
|
|
import {
|
|
PyFrameworksPydanticBaseExample,
|
|
PyFrameworksPydanticBaseModel,
|
|
PyFrameworksPydanticImports,
|
|
PyFrameworksPydanticSetUrl,
|
|
PyFrameworksPydanticTypeConversion,
|
|
PyFrameworksPydanticVectorField,
|
|
} from '/snippets/integrations.mdx';
|
|
|
|
[Pydantic](https://docs.pydantic.dev/latest/) is a data validation library in Python.
|
|
LanceDB integrates with Pydantic for schema inference, data ingestion, and query result casting.
|
|
Using `lancedb.pydantic.LanceModel`, users can seamlessly
|
|
integrate Pydantic with the rest of the LanceDB APIs.
|
|
|
|
First, import the necessary LanceDB and Pydantic modules:
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyFrameworksPydanticImports}
|
|
</CodeBlock>
|
|
|
|
Next, define your Pydantic model by inheriting from `LanceModel` and specifying your fields including a vector field:
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyFrameworksPydanticBaseModel}
|
|
</CodeBlock>
|
|
|
|
Set the database connection URL:
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyFrameworksPydanticSetUrl}
|
|
</CodeBlock>
|
|
|
|
Now you can create a table, add data, and perform vector search operations:
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyFrameworksPydanticBaseExample}
|
|
</CodeBlock>
|
|
|
|
|
|
## Vector Field {#vector-field}
|
|
|
|
LanceDB provides a `lancedb.pydantic.Vector` method to define a
|
|
vector Field in a Pydantic Model.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyFrameworksPydanticVectorField}
|
|
</CodeBlock>
|
|
|
|
This example demonstrates how LanceDB automatically converts Pydantic field types to their corresponding Apache Arrow data types. The `pydantic_to_schema()` function takes a Pydantic model and generates an Arrow schema where:
|
|
- `int` fields become `pa.int64()` (64-bit integers)
|
|
- `str` fields become `pa.utf8()` (UTF-8 encoded strings)
|
|
- `Vector(768)` becomes `pa.list_(pa.float32(), 768)` (fixed-size list of 768 float32 values)
|
|
- The `False` parameter indicates that the fields are not nullable
|
|
|
|
## Type Conversion {#type-conversion}
|
|
|
|
LanceDB automatically convert Pydantic fields to
|
|
[Apache Arrow DataType](https://arrow.apache.org/docs/python/generated/pyarrow.DataType.html#pyarrow.DataType).
|
|
|
|
Current supported type conversions:
|
|
|
|
| Pydantic Field Type | PyArrow Data Type |
|
|
| ------------------- | ----------------- |
|
|
| `int` | `pyarrow.int64` |
|
|
| `float` | `pyarrow.float64` |
|
|
| `bool` | `pyarrow.bool` |
|
|
| `str` | `pyarrow.utf8()` |
|
|
| `list` | `pyarrow.List` |
|
|
| `BaseModel` | `pyarrow.Struct` |
|
|
| `Vector(n)` | `pyarrow.FixedSizeList(float32, n)` |
|
|
|
|
LanceDB supports to create Apache Arrow Schema from a
|
|
`pydantic.BaseModel`
|
|
via `lancedb.pydantic.pydantic_to_schema` method.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyFrameworksPydanticTypeConversion}
|
|
</CodeBlock>
|
|
|
|
This example shows a more complex Pydantic model with various field types and demonstrates how LanceDB handles:
|
|
- Basic types: `int` and `str` fields
|
|
- Vector fields: `Vector(1536)` creates a fixed-size list of 1536 float32 values
|
|
- List fields: `List[int]` becomes a variable-length list of int64 values
|
|
- Schema generation: The `pydantic_to_schema()` function automatically converts all these types to their Arrow equivalents |