The `Reranker` docstring says the default `return_score="relevance"` "will return only the relevance score" (`rerankers/base.py`), but a hybrid search whose filter matches nothing also returns `_distance` and `_score`. The early return for two empty result sets in `_combine_hybrid_results` (from #3030) skips the reranker's `_keep_relevance_score`, which every built-in reranker applies to a non-empty result.
Before, on `45dbf5e`:
```
$ python repro.py
match: ['id', 'text', 'vector', '_relevance_score']
no match: ['id', 'text', 'vector', '_distance', '_score', '_relevance_score']
```
After, `no match` also prints `['id', 'text', 'vector', '_relevance_score']`.
<details><summary>repro.py</summary>
```python
import tempfile, lancedb, pyarrow as pa
db = lancedb.connect(tempfile.mkdtemp())
t = db.create_table("t", pa.table({"id": [1, 2], "text": ["red fox", "blue fox"],
"vector": pa.array([[1.0, 2.0], [3.0, 4.0]], pa.list_(pa.float32(), 2))}))
t.create_fts_index("text")
q = lambda where: t.search(query_type="hybrid").vector([1.0, 2.0]).text("fox").where(where, prefilter=True).limit(5)
print("match: ", q("id > 0").to_arrow().column_names)
print("no match:", q("id > 100").to_arrow().column_names)
```
</details>
No issue exists for this. The fix calls `reranker._keep_relevance_score(empty)` before returning. Sync and async hybrid queries both go through it. With `return_score="all"` both columns stay, as before. Code reading `_distance` from an empty result now gets a `KeyError`, which it already got from every non-empty one. The new test asserts, for both settings, that a filter matching nothing gives the same columns as one matching every row.
### Verification
```
# Python 3.12.6; lancedb 0.41.0b15 wheel from pypi.fury.io, its _lancedb.abi3.so in the tree; no pylance (none >= 15 resolves)
# the wheel predates #4688, so a 3-line -p plugin sets _lancedb._experimental_functions_enabled = False for tests/conftest.py
$ PYTHONPATH=python python -m pytest -q -p no:cacheprovider python/tests/test_rerankers.py python/tests/test_hybrid_query.py python/tests/test_query.py
45dbf5e: 5 failed, 197 passed, 57 skipped
this branch: 5 failed, 199 passed, 57 skipped
query.py reverted, test kept: 6 failed, 198 passed, 57 skipped
FAILED test_rerankers.py::test_empty_hybrid_result_keeps_reranker_columns[relevance]
At index 2 diff: '_distance' != '_relevance_score'
$ ruff format --check <both files> && ruff check <both files> # ruff 0.9.9, as pre-commit pins it
2 files already formatted
All checks passed!
# the 5 failures fail identically on 45dbf5e: 4 test_query_serialization_async_search_options (wheel older than #4659) and
# test_hybrid_query_with_stale_fixed_size_binary_prefilter (needs pylance)
# not run: pyright, the Rust build, the remote client
```
@Xuanwo, a custom reranker that keeps extra columns still gets the generic empty table. I'd keep it that way, since #3030 returns early so that no reranker runs on empty input. If you'd rather call `rerank_hybrid` on the empty tables instead, I'll change it in this PR.
Part of #4364 (Python side; the TypeScript `FtsOptions` change will follow in a separate PR).
Lance already supports the code analyzer, but `FTS` and `lancedb.tokenize()` had no way to set it, so `FTS({'analyzer': 'code'})` and `FTS(split_identifiers=True)` raised `TypeError`.
## Changes
- `FTS` and `lancedb.tokenize()` gain `analyzer` (`"text"` or `"code"`), `split_identifiers`, `split_on_numerics`, `preserve_original` and `index_operators`. They default to `None`, which keeps the tokenizer's own defaults, so existing indexes and calls are unchanged.
- Validation follows pylance: `analyzer="code"` with a `base_tokenizer` other than `"code"` or the default `"simple"` is rejected, `analyzer="text"` with `base_tokenizer="code"` is rejected, and turning on a code flag without the code analyzer is rejected. Passing a flag as `False` is always allowed.
- `analyzer="code"` with the default `base_tokenizer="simple"` selects the code tokenizer. `FTS` can't tell an explicit `"simple"` from the default, so that case is accepted too.
- The deprecated `create_fts_index` is left alone; it points users at `create_index(config=FTS(...))`.
- Rust `FtsIndexBuilder` is Lance's `InvertedIndexParams`, so the Rust SDK already had these options. The remote client serializes the same struct, so the options reach LanceDB Distributed in the `create_index` request body. A new case in the Rust `test_create_index` checks that body.
## Tests
- `python/tests/test_fts.py`: tokenizer output for each flag, validation errors for `tokenize` and `FTS`, `False` flags on a text index, and local sync and async indexes that match `http`, `request` and `parseHTTPRequest` with splitting on and not match subwords with it off.
- `cargo test --features remote remote::table::tests::test_create_index`
Update the Rust workspace and Java SDK dependencies to [Lance
v15.0.0-beta.8](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.8),
preserve the optional `manifest_size` field in remote version responses,
and verify directory-namespace renames preserve registration and data
both within and across namespaces. Update the Rust and TypeScript API
documentation to reflect directory-namespace rename support.
Rust callers constructing `Version` literals must now provide
`manifest_size` (`None` when unavailable). Java is intentionally pinned
to beta.8 even while Maven artifacts are still publishing;
dependency-resolution CI failures caused solely by that publishing delay
are accepted for this upgrade. Python package pins remain unchanged.
---------
Co-authored-by: Jack Ye <yezhaoqin@gmail.com>
## Summary
- Add sync/async `DBConnection.list_gpu_classes()` returning typed
`GpuClass` records for `GET /v1/gpu/classes`. Records include the class
name, Kubernetes resource, GPU model and memory, per-worker GPU maximum,
and worker CPU/memory/ephemeral-storage limits.
- Add `udf(gpu_class=...)` for immutable Function-level GPU placement.
It requires `gpu=True`; Function versions expose their registered
`gpu_class`.
- Add `gpu_class` and `gpu_count` to `refresh_column` /
`refresh_column_async` and Rust `RefreshColumnOptions`. Both are sent as
top-level `backfill_column` fields and validated before any request.
`gpu_count` is the aggregate GPU budget across Function workers.
- Validate GPU class names with the server lowercase 63-character
DNS-label rule.
- Document the discovery-to-registration flow, configured hardware
versus free capacity, System Usage permission, and per-refresh GPU
placement in the API and web guides.
Older servers without `/v1/gpu/classes` produce a clear
`NotImplementedError` instead of a bare 404.
Part of ENT-3470. The client follows the in-flight Sophon contract on
`xuanwo/ent-3470-fix` for GPU class discovery and backfill placement
options.
## Breaking change
`PythonRuntimeSpec::PythonV2` adds `gpu_class`, and
`RefreshColumnOptions` adds `gpu_class` and `gpu_count`. Rust callers
constructing either value with all fields explicitly must supply the new
fields.
## Validation
At `7685c968`: Rust GPU tests (5 passed), Rust/PyO3 clippy, Python
extension build, pyright, ruff, and Python Function/runtime, discovery,
refresh placement, and refresh regressions (147 passed, 6 skipped).
At `b22428fd`: Rust GPU tests (5 passed), rustfmt, ruff, pyright,
Function slice 2 (85 passed), and remote GPU/refresh tests (20 passed).
The final commit only updates documentation and Rust doc comments.
## Summary
- Make `AsyncJob.wait(timeout)` and `Job.wait(timeout)` raise a
descriptive `TimeoutError` containing the job ID and elapsed timeout.
Registry jobs explain that work continues on the server and include the
`db.open_job(id).wait()` reattach call. SQL query handles instead say to
keep the original handle because query IDs cannot be reopened through
the registry jobs API.
- Render `JobFailureInfo` and `FunctionErrorRecord` with Python values
instead of Rust `Option` wrappers.
- Decode and expose the server's `row_address` on `FunctionErrorRecord`,
and document how to retry recorded rows with `REFRESH COLUMN ... WHERE
_rowaddr IN (...)` before compaction or rewrite.
- Warn when an import inside a packaged Function body resolves to
undeclared local source. The `udf` documentation explains why arbitrary
base-image coverage and distribution-to-import-name mapping cannot be
validated reliably by the SDK.
`row_address` is optional, so records from older servers still decode
with `None`. Rust code that constructs `FunctionErrorRecord` literals
must now supply this new field.
Part of ENT-3448 and ENT-3442. This consumes the server-side diagnostics
from lancedb/sophon#9509 and logical row addresses from
lancedb/sophon#9512.
## Validation
- Rust `function_errors` remote decoding and
legacy-missing-`row_address` tests.
- Rust/PyO3 clippy for `lancedb` and `lancedb-python`.
- Python extension build, pyright, and ruff.
- Targeted Python job, Function packaging, Function error, and job
detail tests: 62 passed, plus timeout-handle tests: 3 passed.
## Summary
- Parse structured `error_code` and `resource` fields from remote HTTP
error bodies and carry them through Rust/PyO3.
- Add `SecretNotFoundError`, `FunctionNotFoundError`,
`NamespaceNotFoundError`, and `AlreadyExistsError` as `HttpError`
subclasses.
- Expose `error_code`, `resource_kind`, `resource_name`, and
`namespace_path` on HTTP exceptions while preserving `status_code`,
`request_id`, and existing `except HttpError` handlers.
- Do not retry structured domain errors that would otherwise become
`RetryError` (notably `already_exists` on 409).
Older servers that do not provide the structured fields continue to
raise a plain `HttpError`.
## Compatibility
This consumes the stable resource error contract added in
lancedb/sophon#9516 (merge commit `35858561b3`). The existing
`TableNotFoundError` behavior is unchanged.
Part of ENT-3373.
`list_functions()` promises every published immutable revision. Request
`SHOW FUNCTIONS HISTORY` so the SQL server's latest-only default does
not hide older revisions, and preserve the server's numeric revision
ordering when constructing page cursors (`1`, `2`, `10`).
```python
[(fn.name, fn.version) for fn in db.list_functions(page_limit=2)]
# [("embed", "1"), ("embed", "2"), ("embed", "10")]
```
```sql
SHOW FUNCTIONS; -- latest visible revision per name
SHOW FUNCTIONS HISTORY WITH (limit = 3, include_definition = true);
SHOW FUNCTIONS HISTORY WITH (after_name = 'embed', after_version = '2', limit = 3);
```
Each listing page uses the existing SQL query and polling transport; the
continuation token stores the last returned name/revision and remains
bound to its database and namespace. Function mutations keep their
query-backed job handles: `Job.id` is the query's ID and `job_type` is
`query`, so waiting observes that same SQL execution. No registry job or
second query ID is added.
Update the SQL and Python documentation for exact-revision drops:
sibling revisions and grants survive, and only the last visible
revision's removal releases the name and schedules object cleanup.
Extend the existing Flight SQL regression to paginate revisions `1`,
`2`, and `10` and verify the explicit history statement. Formatting and
static Python checks pass; local builds and tests were intentionally not
run.
## What kind of change does this PR introduce?
Feature. Exposes the NGram scalar index in the Python SDK.
## What is the current behavior?
The NGram scalar index already exists in the Rust core (`Index::NGram`) and is accepted over the remote protocol (the `"NGRAM"` type), but the Python SDK has no way to create it. `lancedb.index` offers `BTree`, `Bitmap`, `LabelList`, `Fm`, and the vector indexes, but not `NGram`, so it cannot be built through `create_index(config=...)`.
## What is the new behavior?
Adds an `NGram` config to `lancedb.index` and wires it through the pyo3 binding:
```python
from lancedb.index import NGram
await table.create_index("text", config=NGram())
```
The NGram index accelerates substring filters such as `contains(col, 'needle')`, along with some `LIKE` and regular-expression predicates, on UTF-8 string columns. It is parameter free and uses Lance's default trigram parameters.
Changes:
- `lancedb.index`: new `NGram` dataclass, added to `__all__`.
- pyo3 binding (`python/src/index.rs`): a match arm mapping `NGram` to `LanceDbIndex::NGram`, plus the import and the error message.
- Type hints: `NGram` added to the `create_index` config unions and the isinstance check in `table.py`, the `IndexConfigType` union, and the `_lancedb.pyi` stubs; `"NGRAM"` added to the `IndexType` and `IndexStatistics.index_type` literals.
- Test: `test_create_ngram_index` covers creation, `list_indices`, `index_stats`, and that a substring filter uses the index.
- Docs: an NGram section in the scalar index guide.
This mirrors the approach used for the RTree index in https://github.com/lancedb/lancedb/pull/4555 (that PR keeps the RTree scope; this one adds the remaining parameter free string index).
Fixes#4749
Python's `__version__` forwards Cargo's SemVer spelling, so prereleases
report values such as `0.41.0-beta.17` while the installed package
metadata reports `0.41.0b17`.
Normalize the native version with the existing
`packaging.version.Version` dependency before exposing it in Python.
Keep native version sourcing so imports still work with `lancedb-compat`
and do not require distribution metadata. Regression tests cover
metadata equality and stable, alpha, beta, release-candidate, and
development versions with metadata lookups disabled.
Validation:
- Rebuilt the native extension through the uv-managed development
environment.
- Confirmed the metadata regression fails on the original code with the
reported string mismatch.
- Import tests, package metadata tests, the remote client user-agent
test, and CI Python dependency tests: 14 passed, 2 skipped (Windows-only
wheel check and uninstalled `lancedb-compat`).
- macOS ARM64/Python 3.13 dependency resolution and `uv lock --check`
pass with the platform-specific test dependency pins from current
`main`.
- Python-tree Ruff lint and formatting checks, focused Pyright check,
Rust formatting, and PR diff whitespace checks pass.
- Repository-wide Ruff lint reports existing findings in unchanged
`docs/web-tests` files.
<!-- lance-gatekeeper-fix:v1 agent=3e23555743cc116ea53ca8a7e3c4ebb1
generation=1 -->
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Fixes#4750
Remote missing-table errors retained HTTP metadata in Rust but the Python binding converted them to a plain `ValueError`, discarding the request ID and status. This raises a `TableNotFoundError` that preserves the original message, request ID, status code, and chained HTTP cause while remaining compatible with existing `ValueError` and `lance_namespace.errors.TableNotFoundError` handlers. Local and non-HTTP errors retain their previous behavior.
Validation covers sync and async opens across cached/uncached and root/nested paths, exception pickling, local compatibility, the remote and error test suites, Clippy, formatting, Ruff, and all applicable PR checks including CodeQL.
`LanceModel` instances with validation aliases write field-name keys to Arrow, while `to_pydantic` previously reconstructed models by alias. This normalizes field-name rows to the applicable single-component validation alias for both sync and async reads, while preserving existing alias keys, alias precedence, name-only validation, and collision behavior.
Adds sync and async regression coverage for string aliases, `AliasChoices`, single-component `AliasPath`, `populate_by_name`, and explicit `validate_by_name` precedence.
Allow OpenAIEmbeddings.ndims() to use an explicitly configured dim for model
names outside the three built-in OpenAI names, so schemas can be built for
compatible models.
Preserve existing model defaults and request behavior. Add regression coverage
for the resulting Arrow schema dimensions and for all known OpenAI model
dimensions.
Allows Python namespace implementations to read request dictionary entries as attributes while preserving mapping and `model_dump()` behavior. Missing dunder attributes continue to raise `AttributeError`.
Adds a real namespace connection regression covering list/open operations and omitted optional fields.
Python column refresh methods rejected `batch_size`, and the Rust remote
backend never included it in the backfill request. Callers could not
reduce the rows passed to a slow Function to stay within the executor's
per-call deadline.
Add optional, keyword-only `batch_size` arguments throughout the
synchronous and asynchronous Python APIs and bindings. Add compatible
Rust `RefreshColumnOptions` APIs while preserving existing Rust method
signatures. Reject non-positive sizes before submitting a job, send
positive sizes in `backfill_column`, and omit the field when unset while
retaining branch scoping.
Batch size controls remote Function calls. Embedded expression refreshes
accept and validate the option without changing execution, and blocking
remote refreshes retain their existing unsupported behavior; remote
callers use `refresh_column_async` and wait on the returned job.
Validation:
- Reproduced the original Python `TypeError` from the committed method.
- Five focused Rust tests passed, covering request contents, omission,
branch scoping, and rejection before any request.
- `cargo check --quiet --features remote --tests --examples` passed.
- `cargo clippy --quiet --features remote --tests --examples -- -D
warnings` passed.
- Nineteen targeted Python tests passed against the rebuilt extension,
including synchronous and asynchronous remote requests, invalid sizes,
and default-build behavior. Eight embedded refresh cases require the
experimental-functions build and were skipped.
- Rust formatting and Python formatting/lint checks passed, including
CI's pinned Ruff version. Repository-wide Ruff reports 48 existing
findings in untouched documentation files; none are in changed files.
Fixes#4776
<!-- lance-gatekeeper-fix:v1 agent=4906ff3f5f7797a1d49a13803302021c
generation=1 -->
---------
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Adds `HUGGINGFACE` and an optional token Secret reference to the
existing storage integration methods. The same SQL transport handles
create, describe, list and drop. Only a Secret reference is stored in
integration metadata; mixed-provider credential options are rejected.
```python
from lancedb.storage_integration import SecretReference
info = db.create_storage_integration(
"private_hub", "hf://datasets/acme/private/raw/",
storage_provider="HUGGINGFACE",
huggingface_token_secret=SecretReference(
"hf_token", database="shared", namespace_path=("credentials",),
),
)
print(info.configuration_instructions)
info = db.describe_storage_integration("private_hub")
names = list(db.list_storage_integrations())
db.drop_storage_integration("private_hub")
```
Omit the token Secret for anonymous access at Hugging Face's default
anonymous
rate limits. A tokenless integration does not use a cluster's Hugging
Face token:
```python
info = db.create_storage_integration(
"public_hub", "hf://datasets/acme/public/raw/",
storage_provider="HUGGINGFACE",
)
```
```sql
CREATE STORAGE INTEGRATION public_hub
STORAGE_PROVIDER = 'HUGGINGFACE'
STORAGE_PREFIX = 'hf://datasets/acme/public/raw/';
```
```sql
CREATE STORAGE INTEGRATION private_hub
STORAGE_PROVIDER = 'HUGGINGFACE'
STORAGE_PREFIX = 'hf://datasets/acme/private/raw/'
HUGGINGFACE_TOKEN_SECRET = shared.credentials.hf_token;
DESCRIBE STORAGE INTEGRATION private_hub;
SHOW STORAGE INTEGRATIONS;
SELECT * FROM 'hf://datasets/acme/private/raw/*.parquet' LIMIT 10;
DROP STORAGE INTEGRATION private_hub;
```
```mermaid
flowchart LR
Python[Sync or async Python] --> SQL[Existing SQL transport]
SQL --> Metadata[Integration and token Secret reference]
```
Adds Rust request construction, SQL rendering, typed metadata decoding,
sync/async Flight SQL regressions and public documentation. Local
builds/tests are excluded by request; CI provides execution validation.
Adds GCP inputs to the existing database-owned storage integration
methods. GCP takes one Secret reference; the Secret stores the complete
application credentials JSON as an opaque string. The same methods and
SQL transport handle create, describe, list and drop.
```python
info = db.create_storage_integration(
"gcs", "gs://customer-data/raw/", storage_provider="GCP",
gcp_credentials_json_secret=SecretReference(
"gcp_json", database="shared", namespace_path=("credentials",),
),
)
print(info.configuration_instructions)
info = db.describe_storage_integration("gcs")
names = list(db.list_storage_integrations())
db.drop_storage_integration("gcs")
```
```sql
CREATE STORAGE INTEGRATION gcs
STORAGE_PROVIDER = 'GCP'
STORAGE_PREFIX = 'gs://customer-data/raw/'
GCP_CREDENTIALS_JSON_SECRET = shared.credentials.gcp_json;
DESCRIBE STORAGE INTEGRATION gcs;
SHOW STORAGE INTEGRATIONS;
DROP STORAGE INTEGRATION gcs;
```
```mermaid
flowchart LR
Python[Sync or async Python] --> SQL[Existing Flight SQL transport]
SQL --> Config[Configuration and one Secret reference]
```
Adds Rust request construction, SQL rendering and typed metadata
decoding, plus sync/async Flight SQL regressions across the three Arrow
string encodings. Rejects missing or mixed-cloud credential options.
Local builds/tests are excluded by request; validation runs in CI.
Register Azure Blob integrations through Flight SQL using an explicit
storage account and three Secret references for tenant ID, client ID and
client secret. Extends the database-owned integration APIs with
provider-specific inputs while keeping the same create, describe, list
and drop operations.
```python
info = db.create_storage_integration(
"customer_blobs", "az://customer-data/raw/",
storage_provider="AZURE", azure_storage_account="customeraccount",
azure_tenant_id_secret="tenant", azure_client_id_secret="client",
azure_client_secret_secret="credential",
)
print(info.configuration_instructions)
info = db.describe_storage_integration("customer_blobs")
names = list(db.list_storage_integrations())
db.drop_storage_integration("customer_blobs")
```
Sync and async connections accept the same options. Use
`SecretReference` for cross-database or namespace qualification. Values
remain in Secrets; returned metadata includes only references and setup
guidance.
```sql
CREATE STORAGE INTEGRATION customer_blobs
STORAGE_PROVIDER = 'AZURE'
STORAGE_PREFIX = 'az://customer-data/raw/'
STORAGE_AZURE_STORAGE_ACCOUNT = 'customeraccount'
STORAGE_AZURE_TENANT_ID_SECRET = public.tenant
STORAGE_AZURE_CLIENT_ID_SECRET = public.client
STORAGE_AZURE_CLIENT_SECRET_SECRET = public.credential;
DESCRIBE STORAGE INTEGRATION customer_blobs;
SHOW STORAGE INTEGRATIONS;
DROP STORAGE INTEGRATION customer_blobs;
```
```mermaid
flowchart LR
Python[Sync or async Python] --> SQL[Flight SQL]
SQL --> Metadata[Configuration and Secret references]
```
Azure supports service principal client credentials only. Adds SQL
rendering, provider validation and Python Flight SQL regressions across
Utf8, LargeUtf8 and Utf8View results. Local builds and tests are
excluded as requested; validation runs in CI.
Add `on_transform_error="fill"` to `StreamingDataset`, replacing failed
transform rows at their original positions with a deep-copied fill value or
the result of `repair(exc, row_batch)`.
Fill mode preserves split length, elastic step layout, worker accounting, and
checkpoint progress. Repair failures remain fatal, and `rows_filled` is shared
across workers.
Validated with focused fill and adjacent dataloader tests (10 and 6 passed)
and the complete applicable CI run for
6b51b33e56: four workflows and fifteen checks
all succeeded.
Closes#3916
Expose table-level maintenance override read, update, and reset APIs for remote
Rust and Python tables. Local tables continue to return `NotSupported`.
Updates recursively merge the caller-provided JSON object into the current
override document so unrelated and unknown service fields are preserved. Reset
uses the explicit reset endpoint.
The integrated change keeps current main's remote database identity, namespace,
branch, SQL, WAL, and threaded mock-server behavior.
Validated with focused Rust and Python override tests and the complete applicable
CI run for 43bccbadc5.
Replaces #4322, keeping only its property graph defined over tables.
A property graph maps node labels and relationship types to existing tables of its namespace, in the shape of SQL/PGQ's `CREATE PROPERTY GRAPH`. It stores only that definition: a query reads the tables as nodes and edges through SQL `GRAPH_TABLE`, as they are when it runs.
- **Rust:** `Connection::create_property_graph` takes a `PropertyGraphDefinition` of `NodeTable`s and `EdgeTable`s. `describe_property_graph`, `drop_property_graph` and `list_property_graphs` manage it.
- **Python:** `lancedb.graph` has `NodeTable`, `EdgeTable` and `Endpoint`. Sync and async connections gain `create_property_graph`, `open_property_graph`, `drop_property_graph` and `list_property_graphs`, and a `PropertyGraph` handle describes its graph.
- **Support:** remote connections only. Local connections raise `NotImplementedError` in Python and return `NotSupported` in Rust.
```python
import lancedb
from lancedb.graph import EdgeTable, Endpoint, NodeTable
db = lancedb.connect("db://demo", api_key="...")
db.create_property_graph(
"social",
nodes=[NodeTable("person", key="person_id", label="Person")],
edges=[
EdgeTable(
"knows",
label="KNOWS",
source=Endpoint("src_id", references=("person", "person_id")),
destination=Endpoint("dst_id", references=("person", "person_id")),
)
],
)
db.execute_query("""
SELECT * FROM GRAPH_TABLE (social
MATCH (p:Person)-[:KNOWS]->(f:Person) WHERE p.age > 30
COLUMNS (p.name AS person, f.name AS friend))
""").read_all()
```
## Problem and change
Some Keycloak client policies require S256 PKCE during device-code
login. Sending those parameters unconditionally breaks interoperability
with providers that expect standard RFC 8628 requests.
Configure device login with `OAuthFlow::DeviceCode(DeviceCodeOptions)`,
following the existing authorization-code options pattern.
`DeviceCodeOptions::new()` keeps PKCE disabled; `.use_pkce(true)` sends
an S256 challenge during device authorization and its matching verifier
during token polling. There is no automatic downgrade, and enabling PKCE
preserves the existing token-cache identity. Python, Node, and CLI
conversions retain ordinary device login without PKCE.
OIDC discovery and token requests now include a versioned User-Agent.
`env!("CARGO_PKG_VERSION")` is expanded at compile time and embeds the
version in the binary; no Cargo environment variable is needed when
running an installed binary or native binding.
The supported-flows guide documents the Rust-only device PKCE option and
its default.
## Rust API migration
BREAKING CHANGE: `OAuthFlow::DeviceCode` now carries
`DeviceCodeOptions`. Replace construction with
`OAuthFlow::DeviceCode(DeviceCodeOptions::new())` or
`OAuthFlow::DeviceCode(Default::default())`, and match with
`OAuthFlow::DeviceCode(_)`. Import `DeviceCodeOptions` from
`lancedb::remote` to opt into PKCE. Python and TypeScript public APIs
are unchanged.
## Validation
- `cargo +1.97.1 test --locked -p lancedb --features remote --lib
remote::`: 679 passed, including PKCE-on/off polling, option defaults,
User-Agent headers, and cache identity.
- `cargo +1.97.1 test --locked -p lancedb-python -p lancedb-nodejs --lib
test_device_code_conversion`: both native-binding conversion tests
passed.
- `cargo +1.97.1 test --locked -p lancedb --features remote --doc
remote::`: 6 passed, including the new options example and updated
session examples.
- `cargo +1.97.1 test --locked -p lancedb-cli --lib auth::tests::`: 11
passed.
- `cargo +1.97.1 fmt --all -- --check` and `git diff --check`: passed.
- A compiled version-string check ran successfully with
`CARGO_PKG_VERSION` unset at runtime.
---------
Co-authored-by: Ryan Haynes <ryan.haynes@lancedb.com>
Fixes#4403
Implement remote drop_all_tables() support by listing tables and dropping them individually, including tables within namespaces. Handle paginated table listings and propagate errors appropriately. Add tests covering root and namespace tables, pagination, empty databases, and listing failures to ensure reliable remote database cleanup.
Adds `image` to `@udf`, setting `runtime.environment.image` so a
Function can be built on a digest-pinned registry image instead of the
default base, for dependencies pip, conda and apt cannot express. The
client refuses references that are not pinned by digest; the server
validates the rest. The field is omitted when unset, so existing
Function identities are unchanged.
Stacked on #4675.
Remote view create, describe, list, and drop now execute SQL. CREATE returns the stored definition, resolution defaults, and Arrow schema from the same execution; DESCRIBE VIEW returns that metadata and SHOW VIEWS supports scoped pagination.
`drop_view_async()` returns a `job_type="query"` handle whose single `job.id` is the SQL query UUID. Waiting drains the query result once and caches completion; `describe_query(job.id)` works on the submitting connection. Completion means catalog removal, with physical cleanup continuing separately. Query handles have no registry history and cannot be reopened through `open_job()`.
Stacked on #4674.
Remote Secret creation, rotation, description, listing, and drop now use the SQL backend. Writes bind the credential to `CREATE SECRET name AS $1` or `ALTER SECRET name AS $1`, keeping values out of SQL text. Description returns metadata; listing uses namespace-scoped continuation tokens.
The existing synchronous APIs submit a SQL query and wait internally through the same query-job adapter as function operations. The single query identity is retained internally; no registry job or extra query ID is introduced. Pollable shared results contain status or metadata and never credential values.
---------
Co-authored-by: Xuanwo <github@xuanwo.io>
Remote function registration, exact-version lookup, listing, and drop now execute SQL through the Flight client. Registration waits for the durable image build and catalog publication before yielding its typed `FunctionVersion`. SQL also covers source-inclusive description and pinned metadata updates.
Registration and drop handles use `job_type="query"`; the single `job.id` is the SQL query UUID and can be inspected with `describe_query()` on the submitting connection. Waiting drains the SQL result once and caches completion for repeated waits. The drop boolean reports catalog removal; physical cleanup continues separately. Query handles have no registry history and cannot be reopened with `open_job()`.
Document vectorized `pyarrow.Array` UDFs for Blob v2 inputs and outputs.
Clarify annotation-selected array execution and the adapter-v1 client/executor
boundary. Add public regression coverage for registering, hydrating, executing,
publishing, and reading back Blob arrays, including large, empty, and null values.
Use `ThreadingHTTPServer` for the shared remote-test mock so concurrent `table.add` requests are not handled serially by the test server.
Validation:
- Reproduced the connection-reset/response-decoding failure against base `13749a3f138fde2647c8bdb8a1c05545026aee57`: 27 targeted runs passed, then the 28th matched the original CI failure.
- The patched target passed 100/100 runs; `test_remote_db.py` passed 261 tests with 2 skipped.
- Ruff 0.9.9 format and lint checks passed.
- Current-head GitHub CI: 19/19 check runs and 5/5 workflow runs succeeded.
Adds database-owned S3 storage integrations with Secret-style namespace
paths in a separate keyspace. Use a regular database under the cluster
root for bootstrap integrations and Secrets. `default` is an example
name, with no enforced or automatically provisioned database; `system`
remains read-only. Authentication uses either an IAM role or two
existing Secret references, including cross-database references;
configuration metadata never contains credential values.
### Python experience
```python
catalog = lancedb.connect_catalog("https://my-server.example", api_key="secret")
db = catalog.connect_database("default")
info = db.create_storage_integration(
"raw", "s3://customer-bucket/raw/",
aws_role_arn="arn:aws:iam::123456789012:role/lancedb-access",
aws_region="us-east-1",
)
print(info.configuration_instructions)
info = db.describe_storage_integration("raw")
print(list(db.list_storage_integrations()))
db.drop_storage_integration("raw")
```
For static credentials, replace `aws_role_arn` with
`aws_access_key_id_secret=SecretReference("s3_key_id",
database="default")` and
`aws_secret_access_key_secret=SecretReference("s3_secret_key",
database="default")`, imported from `lancedb.storage_integration`. Pass
`namespace_path` to address nested namespaces. Async database
connections expose the same APIs; listing is lazy. All four client
operations execute SQL through the existing SQL transport, with typed
Arrow result decoding and name-based listing cursors.
### SQL experience
```sql
CREATE STORAGE INTEGRATION default.public.raw
STORAGE_PROVIDER = 'S3'
STORAGE_PREFIX = 's3://customer-bucket/raw/'
STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::123456789012:role/lancedb-access'
STORAGE_AWS_REGION = 'us-east-1';
DESCRIBE STORAGE INTEGRATION default.public.raw;
SHOW STORAGE INTEGRATIONS IN default.public;
DROP STORAGE INTEGRATION IF EXISTS default.public.raw;
CREATE STORAGE INTEGRATION default.public.raw
STORAGE_PROVIDER = 'S3'
STORAGE_PREFIX = 's3://customer-bucket/raw/'
STORAGE_AWS_ACCESS_KEY_ID_SECRET = default.public.s3_key_id
STORAGE_AWS_SECRET_ACCESS_KEY_SECRET = default.public.s3_secret_key;
```
CREATE and DESCRIBE return configuration and a single sentence of setup
guidance. IAM mode also returns the deployment role principal, stable
external ID, customer trust policy, and prefix-scoped permissions
policy. Registration does not verify access. Creation requires namespace
CREATE_STORAGE_INTEGRATION plus parent USAGE; Secret references require
source database, namespace, and Secret USAGE. Describe requires object
USAGE; drop requires OWNERSHIP. SQL supports GRANT/REVOKE for these
objects and their creation privilege.
Includes client documentation and Flight SQL regression coverage; no
dedicated storage-integration REST endpoints are used. Local
builds/tests are excluded by request.
## Problem
The Python test extra pins `pylance==15.0.0b7` for every platform. On macOS, pip cannot resolve that pin: the configured `lance-format` index publishes b7 wheels for Linux and Windows but not macOS, while public PyPI returns HTTP 404 for the version and the `lancedb` index contains no 15.x release. The macOS Python dependency installation consequently cannot resolve the pin.
## Change
- Keep the exact `pylance==15.0.0b7` test pin on non-macOS platforms.
- Use `pylance==15.0.0b4` on macOS. This is the newest release in this range with a macOS wheel (`cp310-abi3-macosx_11_0_arm64`).
- Refresh `python/uv.lock` for the platform-specific resolutions.
The production Rust/Lance inputs are unchanged: `Cargo.lock` is byte-identical to `main` and remains on Lance `15.0.0-beta.7`. No tests, assertions, coverage, or CI checks are removed or weakened.
## Validation
- Confirmed the wheel identity from the configured index and matched its published SHA-256 to `uv.lock`; wheel metadata reports `Name: pylance`, `Version: 15.0.0b4`.
- Verified the b4 APIs used by the Python tests (`blob_field`, `blob_array`, `BlobType`, `Blob`, `dataset`, `write_dataset`, `FFILanceTableProvider`, and `DirectoryNamespace`). The Python test suite does not reference the b7-only `mark_sstables_as_compacted`, `mem_wal_dir_id`, `mem_wal_index_details`, `clustering_id`, `LanceFragment`, `IndexSegment`, `UpdateConfig`, or `max_source_fragments` changes.
- Verified scalar read/write interoperability in both directions between b4 and the beta.7 LanceDB native binding, plus beta.7 reads of a b4-written blob dataset with byte and null fidelity.
- `uv lock --check` with both configured Fury indexes and `unsafe-best-match`.
- CI dependency tests.
- Ruff 0.9.9 format and lint checks.
- Targeted Python tests: 911 passed, 22 skipped.
- Full Python suite (`-m "not slow and not s3_test"`): 2858 passed, 157 skipped, 70 deselected.
## Summary
Fixes the problems a page-by-page review of the docs found before
launch, after checking each one against the current source and, where
noted, the code. The matching fixes for the dataset and Geneva pages,
which live in `lancedb/docs`, are in lancedb/docs#402.
**Tables**
- `integrations/embedding/cohere`: each input-type description was split
across two table rows and wrapped in literal quotes
(`"`search_document`"`). Now one row per type.
- `integrations/embedding/gemini`: removes the same stray quotes from
the task-type table.
- `integrations/ai/langchain`: the MMR search table's `kwargs` row was
missing a cell, so its description sat under Type.
- Six reranker pages (answerdotai, cohere, colbert, jina, openai,
voyageai): closes the backtick before `_relevance_score`.
- `indexing/vector-index`: finishes a cell cut off at "with scalar".
- `sql/dql/dedupe`: labels the empty header cell in the resolution
table.
**Corrections**
- `curation/sampling`: sampling works in LanceDB Embedded through the
Python SDK, so the page is available in both, with the SQL sections
marked Enterprise.
- `feature-engineering/index`, `add-computed-column`: automatic refresh
is on by default for each computed column and takes effect once a
deployment enables its scheduler. `refresh-column` already said so;
these two implied it was off.
- `indexing/scalar-index`, `indexing/index`: `LABEL_LIST` accelerates
`array_has_all`/`array_has_any`; `array_contains_*` doesn't exist. Also
merges scalar-index's two "nested fields" sections into one.
- `integrations/embedding/ibm`: lists the four current models from
`CURRENT_MODELS` (it listed a deprecated one) and explains that the
default stays a legacy ID so existing tables reload with their model.
- `integrations/reranking/openai`: the class is `OpenaiReranker`, not
`OpenAI()`.
- `integrations/ai/synthetic-data-kit`: the CLI is `synthetic-data-kit`
throughout, each step runs one command, each step passes the file it
writes to the next, and `save-as` uses `-f`/`-o`.
- `integrations/data/dlt`: `secrets.toml` defines `api_token`, which the
pipeline reads, and the run command runs the file the page creates.
- `search/vector-search`: the multivector walkthrough connected to
Enterprise in step 1 and a local path in step 2. It now runs locally
throughout, with a note on connecting to Enterprise, and is renamed so
it doesn't read as a duplicate of the short section above.
**Structure and wording**
- `integrations/embedding/nvidia`: adds headings, and drops test-suite
details aimed at contributors.
- `integrations/ai/genkit`: makes the retrieval test a real heading and
describes the query step.
- `integrations/data/voxel51`: replaces a raw MkDocs `!!! note` with a
`<Note>`.
- `sql/ddl/table`, `sql/ddl/job`: "Errors and limits" is a real heading,
not bold text.
- `curation/windowing`: sidebar title matches the curation overview
("Rank and Deduplicate").
- `deployment/storage`, `basics/agentic-skills`: fixes a garbled
sentence and a stale reference to the quickstart.
**Checked:** no new Vale alerts on the changed pages. Assembled together
with lancedb/docs#402, the site passes `mint validate` and `mint
broken-links --check-anchors --check-redirects`. Every changed page was
reviewed in a local preview.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Updates Rust workspace and Java Lance dependencies to
[v15.0.0-beta.7](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.7),
and pins Python tests to the published `pylance==15.0.0b7` release with
refreshed Cargo and uv lockfiles. No compatibility fixes were required.
Validation: workspace Clippy with all features and warnings denied,
Cargo formatting, uv lock consistency, and all 18 CI-script tests pass;
repository-wide Ruff diagnostics are pre-existing.
---------
Co-authored-by: Lu Qiu <lu@lancedb.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Updates Rust and Java Lance dependencies to
[v15.0.0-beta.6](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.6),
pins Python tests to published pylance 15.0.0b6, and refreshes both
lockfiles.
Aligns Arrow 59, DataFusion 55, and PyO3 0.29; adapts execution plans
and MemWAL directory handling, updates the ABI metadata test, and
removes the obsolete Lance Arrow patch now fixed upstream.
Validation: workspace clippy with all features/tests and warnings
denied, cargo fmt, Python/CI Ruff checks, uv lock check, 16 CI-script
tests, and targeted Python regressions (54 passed, 28 skipped);
repository-wide Ruff retains existing documentation-example failures,
and the upstream macOS wheel build failed while Linux and Windows wheels
are published.
---------
Co-authored-by: Daniel Rammer <hamersaw@protonmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Fixes#4733
PyArrow expects Blob V2 storage structs when constructing a record batch
and cannot infer the extension array from raw bytes. LanceDB already
converts binary values during ingestion, but batch writers had no
supported array constructor.
Add `lancedb.schema.blob_array(payloads)` for iterables of inline bytes
and nulls, using the same extension type resolver as `blob()`. The
helper preserves empty blobs and null blobs and works with both
pylance's type and LanceDB's fallback. Document the descriptor fields,
exclusive inline/external representations, batch construction, and
existing binary conversion in synchronous and asynchronous
`create_table` / `add` APIs.
Validation:
- Reproduced the reported `ArrowTypeError` on the original code; both
new regression cases fail against the original schema module.
- 256 focused blob tests and schema doctests passed; one optional PyAV
test skipped.
- Both new regression cases also passed with PyArrow 25, with and
without pylance imports.
- Python package Ruff checks and formatting passed.
- MkDocs build passed, with an existing unrelated `SecretReference`
cross-reference warning. Repository-wide Ruff reports 48 existing errors
in `docs/web-tests`.
<!-- lance-gatekeeper-fix:v1 agent=20d1a296d76ff5eb914c0848428a747a
generation=1 -->
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Adds a Python `nvidia` embedding provider for Nemotron 3.5 Embed 1B and 8B. It works with NVIDIA's hosted API and self-hosted NIM `/v1` endpoints, and uses the existing `openai` client. The OpenAI provider can't send NVIDIA's passage/query input types or pick Nemotron's dimensions.
- Documents are embedded as `passage`, searches as `query`.
- Default dimensions are 2048 (1B) and 4096 (8B), and `dim` is checked against each model's supported range. Self-hosted model IDs need an explicit `dim`.
- Response count, order, dimensions and values are checked. Null and empty text get a null vector without a request.
- Each request is retried on its own with LanceDB's retry policy. Invalid input or responses fail immediately.
- `NVIDIA_API_KEY` is only sent to the hosted API. Custom endpoints need an explicit `api_key` (a `$var:` reference), and endpoints restored from table metadata must be allowed with `allow_endpoint`.
- Request dimensions are sent through `extra_body`, preserving the NVIDIA wire contract while supporting the declared minimum `openai==1.6.1`.
Python and text only. Docs are in `integrations/embedding/nvidia`.
Originally authored by Taylor (@tsmithv11).
### Testing
- `test_nvidia_embeddings.py`: 45 passed, 2 deselected with both the locked `openai==2.44.0` and minimum `openai==1.6.1`. These use the real OpenAI client with mock HTTP transport and real local tables.
- Embedding, registry, field-metadata and trust suites: 163 passed, 4 skipped, 10 deselected.
- Historical live experiments: hosted 8B passed a five-request smoke test; hosted 1B returned HTTP 500. No live request was made in this stage, so current hosted 1B availability was not verified. Enterprise coverage uses the repository mock HTTP server, not a live deployment.
Add a typed Python `CombinedFieldsQuery` for BM25F search across multiple columns.
The query mirrors Lance's combined-fields semantics with ordered per-column boosts and AND/OR term matching. Convert it in both directions at the PyO3 boundary, including nested Boolean queries, and document the Python typed convenience entry.
Closes#4652.
Support `time` and `timedelta` values in `table.update` by generating explicit Arrow casts.
- Preserve Arrow duration/time units and pandas nanosecond precision.
- Reject timezone-aware `time` values before data can be changed.
- Cover time64 us/ns, duration s/us/ns, negative timedeltas, filters, and nanosecond round trips.
Adds `apt` to `@udf` and `system_packages` to the Rust `PythonEnvironmentSpec`. The server adds the packages to the Function's image before installing its Python environment, for shared libraries that pip and conda cannot provide, such as FFmpeg's for torchcodec. The field is omitted when empty, so existing Function identities are unchanged.
Closes#1605.
Adds `EmbeddingFunctionRegistry.list_functions()` to expose registered lookup names in registration order without accessing `_functions`. The method returns an independent snapshot, and every returned name can be passed to `get()`.
Tests cover empty registries, aliases and class names, registration order, snapshot isolation, and reset.
Map dynamic projections by output name to their source columns in the remote
torch test mock so permutation take queries preserve the aliased ordering
column. Run the regression in the embeddings-installed Python test job and
require torch to import so a missing dependency cannot skip coverage.
The Python test jobs spend much of their pytest time waiting on the in-process mock HTTP servers.
1. Each server ran `serve_forever()` with the default 0.5 s poll interval, so every `server.shutdown()` blocked for up to 0.5 s. Around 300 tests use such a server (test_remote_db, test_authz/test_catalog, test_listing, the permutation helpers), about 150 s per test job on every platform. A shared `start_server_thread()` helper in `tests/utils.py` serves with a 0.01 s poll interval, so shutdown returns in about 10 ms.
2. The servers bind IPv4 only, but clients used `http://localhost:<port>`. The client tries ::1 first and falls back after the happy-eyeballs delay, about 0.3 s per connection on Windows. The mock handlers close the connection after each response, so one listing test took 9.5 s on Windows CI versus 0.6 s on Linux. Clients now use 127.0.0.1, as test_catalog, test_listing and test_namespace already do.
Measured on Windows with the 0.41.0b6 wheel and `-m "not slow and not s3_test"`: full suite 546.7 s to 212.3 s; test_remote_db.py, test_listing.py and test_catalog.py together 284 s to 14 s on current main. Collected test ids are identical (1898) and pass/fail/skip results match. No assertion, test or marker changed.
## Summary
- cover vector-index prewarming through the Python API using the logical
index name returned by list_indices
- prevent regression to interpreting a vector index name as a schema
column
## Root cause
The Lance version used by the reported release opened a vector index by
treating the prewarm argument as a column before resolving the index
metadata. Current LanceDB dependencies contain the upstream
metadata-based resolution, but LanceDB only exercised prewarming for FTS
indexes, leaving the vector behavior unguarded.
## Validation
- uv run --extra tests pytest
python/tests/test_index.py::test_create_vector_index -q
- python/.venv/bin/ruff format --check python/python/tests/test_index.py
- python/.venv/bin/ruff check .
Fixes#2897
<!-- lance-gatekeeper-fix:v1 agent=c2f8273221061d8e2b6f7393066e68c6
generation=1 -->
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
Closes#3693.
`columns` is documented on `StreamingDataset` now, but `connection_factory` still wasn't. This adds it to the parameter list, in signature order.
The text follows `__getstate__` / `__setstate__`: with a factory, pickling the dataset replaces the table's reopen state with its name, and each DataLoader worker calls the factory to reopen the table. The factory is pickled along with the rest of the dataset, so it has to be picklable, and credentials stay out of the pickle only when the factory loads them in the worker instead of capturing them (the same guidance as `Permutation.with_connection_factory`). Without a factory, the table's own reopen state is pickled.
This picks up #3697, which documented the same parameter and was closed by its author without a merge.
Binding a nullable column needed an explicit input schema, and that
forced spelling out the output schema too. Either schema can now be
given alone; the other side is inferred from annotations.
The udf docstring now says a bare annotation is non-nullable and
`Optional[T]` binds a nullable column, and that a `bytes` input counts
against the per-row input limit, with Blob V2 and `BlobReader` for large
values.
Function authors annotate inputs with lancedb.BlobReader, but the
packaged source binds that annotation to the worker's
lance_blob.BlobReader, so grepping the artifact for the client class
finds nothing. Say so on the class.
`@udf` documented batch annotations without explaining that `pa.Array`
and `pa.RecordBatch` do not specify concrete Arrow element types or
field schemas. Document the requirement for both explicit schemas
(including class `__call__` methods), add an array example, and make
inference errors point callers to `input_schema` and `output_schema`.
Fixes#4721
Validation:
- Five new regression cases pass; all four diagnostic cases fail with
the original implementation.
- Function packaging, first-class Function slice 1/2 tests, and UDF
doctests: 168 passed, 3 failed. This includes the upstream BlobReader
annotation coverage. The three existing packaging tests fail while
importing temporary modules, before UDF registration; all three failures
also reproduce with the unchanged source and tests at `057a39a1`.
- Rebuilt the native extension in the uv-managed tests/dev environment.
- Python lint and formatting checks pass (`ruff check python`, `ruff
format --check python`); `cargo fmt --all` also passes without changes.
- Repository-wide `ruff check .` reports 48 existing errors under
`docs/web-tests`.
<!-- lance-gatekeeper-fix:v1 agent=2c6567ee3f128d3a0c3b9c68e36eca38
generation=1 -->
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Remote Python connections inherited the unsupported-operation stubs for
`namespace_exists` and `table_exists`, so both raised
`NotImplementedError` even when namespace and table operations worked.
Add the missing remote overrides, async methods, and PyO3 bindings using
the existing namespace describe and table open paths. Only an explicit
not-found result returns `False`; permission, transport, and server
errors propagate. This preserves authentication, dynamic headers,
retries, and nested namespace handling, and rechecks cached tables
before reporting that they exist.
The async namespace helper also recognizes native `NamespaceNotFound`
errors so missing local namespaces return `False`. The table and
namespace guides document Python remote support and distinguish it from
the unsupported `exists` routes used by direct REST namespace clients.
Validation:
- All 40 focused regression cases pass, covering successful and missing
lookups, local and remote namespaces, dynamic headers, permission/server
errors, invalid table identifiers, and cached tables that disappear.
- The original inherited stubs reproduce the reported
`NotImplementedError`; the local namespace regression reproduced the
native missing-namespace `RuntimeError` before its fix.
- Full remote connection suite: 255 passed. Local namespace tests: 14
passed.
- `cargo clippy --quiet -p lancedb-python --features remote --lib`,
`cargo fmt --all`, Python Ruff lint, and Python formatting checks pass.
- Root `ruff check .` reports 48 existing errors in documentation
examples under `docs/web-tests`; the Python subtree passes and those
unrelated files are unchanged.
Fixes#4722
<!-- lance-gatekeeper-fix:v1 agent=4d369a0681756fdb5a8b35853d7f5607
generation=1 -->
---------
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Function descriptions include `source_digest`, but both the Rust
response model and Python `FunctionVersion` discarded it. Clients
therefore could not compare a catalog entry with
`udf_def.registration_request.artifact.digest`.
Preserve the digest through Rust serialization and expose it on Python
`FunctionVersion`, with a documented Rust accessor. Missing or null
digests decode as `None`, keeping older responses compatible. Regression
tests cover serialization, immutability, registration jobs, get, and
synchronous/asynchronous list responses while keeping the image digest
independent.
Validation:
- Confirmed that the original Python model silently discarded a
server-provided source digest.
- Rebuilt the Python extension in the uv-managed environment; both
focused Function test files passed (106 tests).
- Focused Rust Function model tests passed (10 tests).
- Full workspace `cargo check --quiet --features remote --tests
--examples` and `cargo clippy --quiet --features remote --tests
--examples` passed.
- Rust formatting and Ruff checks/formatting for changed Python files
passed.
- Repository-wide Ruff reports 138 existing violations in unchanged
docs/CI files.
Fixes#4723
<!-- lance-gatekeeper-fix:v1 agent=c145f25136f7b908217e04bde8a2548b
generation=1 -->
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Christopher Krah
SHAIK SHIVAJI
Lance Release
LanceDB RobotandJack Ye
Xuanwo
BHAWESHBHASKAR
lancedb-gatefixer[bot]
Hien Hoang
alexcastgonz
Yui Numazawa
importcpp
Bruno Ramirez
ryan-haynesandRyan Haynes
somil jain
Wyatt Alt
Brendan ClementandClaude Opus 5.5
Taylor
Jay Pokale
Ishita Singh
Lei Xu
max
Bilal Atique