## Summary
- Complete the SQL reference: `ATTACH DATABASE`, the `CHANGES` clause,
and
`dedupe()`/`duplicate_pairs()` (verified against current sophon source;
these were recently renamed from
`vector_dedup`/`vector_duplicate_pairs`).
Fix the blob-column default-projection claim and add the previously
undocumented `blob_descriptor()`.
- Enrich the gated Functions draft (still `.mintignore`'d, pre-release):
chained Functions over blob outputs, GPU-produced blobs, Secrets,
multi-input/output binding, `code=[module]` packaging.
- Upgrade the materialized-view and computed-column auto-refresh drafts
with
precise, verified catch-up semantics, and add the missing `ALTER
MATERIALIZED VIEW` / `CREATE ... WITH(...)` sections (also gated — the
automatic-refresh scheduler is still dev/staging-only).
- Thread one running example (video frames → GPU extract/caption → dedup
→
auto-refreshing materialized view) through Functions, materialized
views,
and the LeRobotDataset integration guide instead of a separate tutorial.
- Rename "LanceDB OSS"/"OSS" to "LanceDB Embedded"/"Embedded" throughout
(52 files); activate the existing availability-comparison mechanism on
`basics/offerings.mdx` for a full feature-by-feature comparison table.
Remote Python connections inherited the unsupported-operation stubs for
`namespace_exists` and `table_exists`, so both raised
`NotImplementedError` even when namespace and table operations worked.
Add the missing remote overrides, async methods, and PyO3 bindings using
the existing namespace describe and table open paths. Only an explicit
not-found result returns `False`; permission, transport, and server
errors propagate. This preserves authentication, dynamic headers,
retries, and nested namespace handling, and rechecks cached tables
before reporting that they exist.
The async namespace helper also recognizes native `NamespaceNotFound`
errors so missing local namespaces return `False`. The table and
namespace guides document Python remote support and distinguish it from
the unsupported `exists` routes used by direct REST namespace clients.
Validation:
- All 40 focused regression cases pass, covering successful and missing
lookups, local and remote namespaces, dynamic headers, permission/server
errors, invalid table identifiers, and cached tables that disappear.
- The original inherited stubs reproduce the reported
`NotImplementedError`; the local namespace regression reproduced the
native missing-namespace `RuntimeError` before its fix.
- Full remote connection suite: 255 passed. Local namespace tests: 14
passed.
- `cargo clippy --quiet -p lancedb-python --features remote --lib`,
`cargo fmt --all`, Python Ruff lint, and Python formatting checks pass.
- Root `ruff check .` reports 48 existing errors in documentation
examples under `docs/web-tests`; the Python subtree passes and those
unrelated files are unchanged.
Fixes#4722
<!-- lance-gatekeeper-fix:v1 agent=4d369a0681756fdb5a8b35853d7f5607
generation=1 -->
---------
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
## Summary
Adopts the product terminology guide across the docs site pages, SDK
docstrings and error messages. "LanceDB OSS" and local mode become
**LanceDB Embedded**. "Remote" as the execution model becomes **LanceDB
Distributed**, which LanceDB Enterprise provides. Where free vs paid is
the point, pages contrast "LanceDB Embedded (open source)" with "LanceDB
Enterprise". This is a first trial of the Embedded name, so reviewers
can judge it in context.
- **Embedded vs Distributed** for where execution runs (`optimize()`,
consistency, branching, maintenance). Main pages define it once:
"LanceDB Enterprise runs LanceDB Distributed: …"
- **LanceDB Cloud** is kept only for legacy-Cloud specifics (`db://`
region, the brute-force KNN limit) and marked legacy. "Cloud and
Enterprise" becomes Enterprise or Distributed.
- **Plan executor** becomes "executor" or the fleet name.
- **Error messages** change, e.g. `optimize is not supported for remote
tables.` → `... in LanceDB Distributed.` Tests that match on them are
updated.
- **Vale rules** (`EmbeddedName`, `OSSProduct`, `RemoteExecution`,
`DistributedEnterprise`, `LegacyCloud`, `InternalNames`) enforce the
terms in prose.
Unchanged: identifiers (`availability.oss`, the `remote` module and
feature, `RemoteTable`, `db://`), heading anchors, internal code
comments, and Geneva.
## Evidence
- Rust lib tests for the changed messages and the Python tests in
`test_remote_db.py`, `test_materialized_views.py` and `test_table.py`
pass locally.
- Not run locally:
- The Node tests: `remote.test.ts` and `table.test.ts` have updated
assertions.
- Vale.
- The `experimental_functions`-marked materialized-view tests.
- `docs/src` API reference was hand-edited to match the docstrings
rather than regenerated.
## Merge Risk
**Door:** two-way; prose and message text only.
**Blast Radius:** `docs`, `error-messages`: callers matching on old
error strings would break.
Companion PRs: lancedb/docs (assembler labels, STYLE.md) and sophon
(Enterprise pages).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The Functions page only showed a plain function, so it never said where a class's `__init__` runs. Users loading a large model need to know it is not the client: registration reads the signature and ships the source.
This adds a section on class Functions: `__init__` runs once per instance on the worker, then `__call__` per row or batch and `close()` on retirement, state is per instance, and `__init__` parameters are initialization fields set per binding. The example's test asserts that registering the class does not construct it.
Plain local blob v2 scans tried to open a pylance dataset before
reaching LanceDB's byte-fetch path, so `to_pandas(blob_mode="bytes")`
failed when the optional dependency was absent. Batch readers also had
no `blob_mode` option.
Route supported blob v2 pandas conversions through the native Arrow
path. Add `blob_mode` to sync and async batch readers, convert payloads
during iteration, and expose the converted schema without fetching the
first batch. Apply conversion after reranking and retain the default
descriptor behavior. Clean up scanner error punctuation and remove the
misleading plain-scan suggestion.
Bytes mode retains the existing `to_arrow` restrictions for legacy/list
blobs, remote tables, and namespace query pushdown. Lazy pandas handles
and legacy pandas scans still require pylance; sync FTS/hybrid batching
remains unsupported.
Validation:
- UV-managed editable Python development build completed successfully.
- The absent-pylance regression tests fail against the original query
implementation and pass with this change.
- Blob and query suites: 403 passed, 1 skipped, 1 deselected.
- Remote query suite: 190 passed.
- Python-tree Ruff lint/format checks and the async reader Pyright check
pass. Repository-wide Ruff reports existing errors outside the Python
tree.
Fixes#4645
<!-- lance-gatekeeper-fix:v1 agent=fd85998f14bb0b3c92b2d050df93a615
generation=1 -->
---------
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
Infer Blob V2 input access from the Python Function signature. `video:
BlobReader` requests a lazy, seekable reader; `video: bytes` requests
the complete payload. Users do not need a separate `blob_inputs`
registration option.
## Python experience
For a video table joined to a table of frame timestamps, the Function
can pass its reader directly to a media decoder:
```python
import io
import av
from lancedb import BlobReader, udf
@udf(pip=["av", "Pillow"])
def extract_frame(video: BlobReader, timestamp_s: float) -> bytes:
with av.open(video) as container:
stream = container.streams.video[0]
container.seek(int(timestamp_s / stream.time_base), stream=stream)
for frame in container.decode(stream):
if frame.time is not None and frame.time >= timestamp_s:
output = io.BytesIO()
frame.to_image().save(output, format="JPEG")
return output.getvalue()
raise ValueError("No frame at or after the requested timestamp")
```
After registering `extract_frame`, the query keeps the usual Function
call:
```sql
SELECT f.video_id, f.timestamp_s,
extract_frame(v.video, f.timestamp_s) AS image
FROM videos AS v
JOIN frames AS f ON v.id = f.video_id
```
The returned JPEG can feed a subsequent image-model step. The reader
provides `read`, `readinto`, `seek`, `tell`, `size()`, and
`read_range(offset, length)`; range reads leave the sequential cursor
unchanged. Each row invocation owns its reader, which closes after the
call. Decoder seeks can request the parts of the MP4 needed to locate
and decode the requested frame; the amount read depends on the file
layout and codec.
Functions that need the whole value can keep a bytes signature:
```python
@udf
def video_length(video: bytes) -> int:
return len(video)
```
## Execution contract
- Signature inference maps `BlobReader` to a Blob V2 field and records
`blob_access="reader"`. `bytes` keeps its binary Arrow type and records
`blob_access="bytes"`; it can also bind to a Blob V2 source. Explicit
schemas support the same choice, including a Blob V2 field annotated as
`bytes`.
- Reader execution carries a compact blob reference into the Function
worker. Payload reads happen when Python calls the reader, with requests
split into at most 1 MiB ranges. The full video does not need to enter
the Arrow input batch just to invoke the Function.
- Bytes execution materializes the full source before the Function runs
and remains subject to eager-input memory admission. Calling
`reader.read()` without a size also materializes the remaining contents
in Python; lazy access depends on how the Function consumes it.
- Input access metadata survives Python/Rust Function serialization and
registration. Packaged Functions import the worker-provided reader
implementation. This PR defines the SDK contract; execution requires
corresponding server support.
- This changes blob inputs. The example still returns ordinary image
bytes; it does not add streamed outputs or run an image model.
Adds coverage for signature inference, explicit schemas, nullable
readers, signature round trips, seeking, bounded range reads, buffering,
and invalid responses.
Document explicit Managed Blob v2 opt-in, descriptor queries and payload
fetching, Function inputs and outputs, and whole-object cleanup. Explain
null versus empty values, snapshot ownership, packed-object retention,
and the distinction from the legacy large-binary Blob API.
The local ingestion and Enterprise Function examples have been executed
end to end. The Function lifecycle description depends on
lance-format/lance#10176 and the corresponding Enterprise migration. The
existing Functions page remains outside the published navigation until
launch.
Validation: the 347-file assembled site has no broken links. Mintlify
4.2.888 build validation reports four existing
`RsExampleBody`/`RsExampleTail` import warnings in the reranking and
vector-search pages; the same diagnostics reproduce with the unchanged
base versions of all three pages edited here.
The existing docs have 254 findings from our shared Vale rules.
Contributors would otherwise inherit these when editing the same lines.
Clean up the prose across 80 pages: correct product names and a repeated
word, simplify a few phrases, and replace dashes with punctuation that
fits each sentence. The pinned Vale scan now reports zero findings, with
no suppressions.
This PR is part of the documentation revamp. Once the full set of PRs is
merged, we’ll publish the combined site to staging for review before
updating docs.lancedb.com.
The curation, SQL and operations pages contain outdated claims and
examples that do not reliably work as written.
This corrects those pages against the reviewed SDK and local server
evidence, including prepared statements, background jobs, cleanup and
monitoring. Integration guides are grouped with their examples in #4533.
Function and Secret-object documentation stays unpublished until its
release. The two Function pages are listed in `.mintignore` and left out
of the navigation, and the sections about them on other pages are MDX
comments.
Stacked on #4533. Each MDX file has one PR owner. Rebase onto main after
#4533 is squash-merged.
Local assembly, Mint validation and link checks pass, and the assembled
site shows no Function or Secret-object content. A local release
rehearsal that uncomments the drafts and restores the navigation also
passes. Hold publication until this PR and lancedb/sophon#8688 are on
`main`.
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).
This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.
Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.
**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.
**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.
Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.
**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.
The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.
---------
Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
Jack Ye
lancedb-gatefixer[bot]
Will JonesandClaude Opus 5.5
Wyatt Alt
Dan Tasse
Drew
Brendan Clement