mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-26 05:26:14 +00:00
fix(python): support typed embedding vector fields (#3826)
## Summary - infer fixed-size float32 Arrow vectors from `VectorField` metadata when a model uses the standard `list[float]` annotation - document the mypy-compatible embedding-field declaration while preserving the existing `Vector(dim)` runtime API - add regression coverage for schema conversion and model construction ## Root cause `Vector(dim)` is a runtime constrained-type factory, but Python's static typing rules do not permit call expressions in annotation positions. Mypy's suggested `Vector[...]` spelling also cannot represent a dimension computed at runtime and fails because `Vector` is a function. `VectorField` already carries the embedding function and its dimension, but schema conversion previously ignored that metadata for a statically valid `list[float]` annotation. The runtime factory remains available for dimension-constrained Pydantic validation. Embedding-backed models that require mypy compatibility can now use `list[float] = embedding.VectorField()`; LanceDB derives the same nullable fixed-size float32 Arrow type from the field metadata. ## Validation - `cd python && uv run --extra tests pytest python/tests/test_pydantic.py -q` (23 passed) - focused mypy check reproducing the original error and accepting the new `LanceModel` declaration - isolated schema checks with Pydantic 1.10.24 and 2.11.7 - `ruff format .` - `ruff check .` Fixes #1982 <!-- lance-gatekeeper-fix:v1 agent=877facbc659cdc9424bd367e12426366 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
This commit is contained in:
@@ -9,6 +9,7 @@ from typing import List, Optional, Tuple
|
||||
import pyarrow as pa
|
||||
import pydantic
|
||||
import pytest
|
||||
from lancedb.conftest import MockTextEmbeddingFunction
|
||||
from lancedb.pydantic import (
|
||||
LanceModel,
|
||||
MultiVector,
|
||||
@@ -425,6 +426,25 @@ def test_bare_vector_raises_clear_error():
|
||||
exec("class TestModel(LanceModel):\n vector: Vector", namespace)
|
||||
|
||||
|
||||
def test_embedding_vector_list_annotation():
|
||||
embedding = MockTextEmbeddingFunction.create()
|
||||
|
||||
class StaticTypingModel(LanceModel):
|
||||
text: str = embedding.SourceField()
|
||||
vector: list[float] = embedding.VectorField()
|
||||
|
||||
schema = pydantic_to_schema(StaticTypingModel)
|
||||
assert schema == pa.schema(
|
||||
[
|
||||
pa.field("text", pa.utf8(), False),
|
||||
pa.field("vector", pa.list_(pa.float32(), embedding.ndims()), True),
|
||||
]
|
||||
)
|
||||
|
||||
model = StaticTypingModel(text="hello", vector=[0.0] * embedding.ndims())
|
||||
assert model.vector == [0.0] * embedding.ndims()
|
||||
|
||||
|
||||
def test_fixed_size_list_field():
|
||||
class TestModel(pydantic.BaseModel):
|
||||
vec: Vector(16)
|
||||
|
||||
Reference in New Issue
Block a user