docs: replace pylance calls with native lancedb equivalents (#4811)

Replace the remaining `pylance` (`take_blobs`) snippet in the LeRobot
integration guide with `Table.fetch_blob_files`, document how to open
`.lance` datasets directly with `lancedb` alongside a
`pylance`-to-`lancedb` call mapping on `/basics/lance`, and clarify in
installation docs that the optional `pylance` extra is only needed for
engine/connector authors or format-level fragment APIs.
This commit is contained in:
Xuanwo
2026-11-04 00:58:05 +08:00
committed by GitHub
parent 8d10f25121
commit 28ba34998a
4 changed files with 44 additions and 7 deletions
+33
View File
@@ -34,12 +34,45 @@ those fields together.
- **Independent column updates.** Add embeddings or other derived features
without rewriting the existing image, video or text columns.
## Opening Lance datasets in Python {#python-packages}
Any `.lance` dataset directory is a LanceDB table. Pass the parent directory to
`lancedb.connect()` and open the table by its directory stem:
```python
import lancedb
db = lancedb.connect("/data") # pass storage_options=... for S3, GCS or Azure
tbl = db.open_table("frames") # opens /data/frames.lance; version=N pins a version
```
Use `lancedb` for search, retrieval, data curation, multimodal blob access and
application code. Install `pylance` (`import lance`) only when building query
engines or connectors, calling format-level fragment APIs directly, or
converting plain-scan lazy or legacy blob columns with `to_pandas()` (see
[Multimodal data and blobs](/management/table#multimodal) for `pylance`-free
conversions).
| `pylance` (`import lance`) | `lancedb` (`import lancedb`) |
| --- | --- |
| `lance.dataset("/data/frames.lance")` | `lancedb.connect("/data").open_table("frames")` |
| `lance.write_dataset(data, "/data/frames.lance")` | `lancedb.connect("/data").create_table("frames", data)` |
| `lance.vector.vec_to_table(mat)` | `db.create_table(name, pa.table({"vector": list(mat)}))` |
| `lance.blob_field("video")` | `lancedb.blob("video")` |
| `ds.take_blobs("video", ids=row_ids)` | `tbl.fetch_blob_files("video", row_ids)` (seekable handles) or `tbl.fetch_blobs("video", row_ids)` (`pa.LargeBinaryArray`) |
| `ds.to_table(columns=cols, filter=expr, limit=n)` | `tbl.search().where(expr).select(cols).limit(n).to_arrow()` |
| `ds.to_batches(batch_size=k)` | `tbl.search().to_batches(k)` |
| `ds.take(offsets)` | `tbl.__getitems__(offsets)` (preserves offset order) or `tbl.take_offsets(offsets).to_arrow()` (unordered) |
| `ds.versions()`, `ds.checkout_version(v)`, `ds.restore()` | `tbl.list_versions()`, `tbl.checkout(v)`, `tbl.restore()` |
| `ds.optimize.compact_files(...)`, `ds.cleanup_old_versions(...)` | `tbl.optimize(compaction_options=..., cleanup_older_than=...)` |
## Learn more {#next}
<span id="versioning" />
<span id="evolution" />
- [Lance format specification](https://lance.org/format): how the format works.
- [Multimodal data and blobs](/management/table#multimodal): declare `lancedb.blob()` columns and read blobs on demand.
- [Table versioning](/management/table#versioning): read and manage table versions.
- [Computed columns](/feature-engineering/add-computed-column): add derived features to your data.
- [Architecture](/index#architecture): how LanceDB Embedded and Enterprise build on Lance.
+2 -2
View File
@@ -126,14 +126,14 @@ tbl.add_columns({"jerk_score": jerk_score}) # nothing computed yet
tbl.backfill("jerk_score", concurrency=8) # 610,403 rows in 16.7 s
```
Embeddings work the same way, reading frames straight from the video blobs in the same table:
Embeddings work the same way, reading frames straight from the video blobs in the `videos` table:
```python
@udf(data_type=pa.list_(pa.float32(), 768), num_gpus=1,
input_columns=["episode_index", "frame_index"])
class EmbedFromBlob: # SigLIP2 loads once per worker
def __call__(self, episode_index, frame_index):
blob = self.videos.take_blobs("video_bytes", indices=[vrow])[0]
blob = self.videos.fetch_blob_files("video_bytes", [video_row_id])[0]
frame = VideoDecoder(blob).get_frames_at([pos]).data[0]
...
```
+4 -2
View File
@@ -63,8 +63,10 @@ uv add --prerelease allow --index https://pypi.fury.io/lancedb/ lancedb
```
</CodeGroup>
If you also need the optional `lance` Python API, install the `pylance` extra and
include the Lance preview index:
Most applications only need `lancedb`. Install the optional `pylance` extra and
include the Lance preview index only when building query engines or connectors,
calling format-level `lance` APIs directly, or converting plain-scan lazy or
legacy blob columns with `to_pandas()`:
<CodeGroup>
```bash Python (pip) icon="terminal"
+5 -3
View File
@@ -20,7 +20,7 @@ For those hosts, install the `lancedb-compat` package instead:
pip install lancedb-compat
```
Same Python API (`import lancedb` works as usual). The compat wheel is compiled at the `x86-64-v2` baseline (Nehalem-class) and uses runtime SIMD dispatch in the embedded lance crate to pick the right kernel tier (scalar / AVX / AVX+FMA / AVX2+FMA / AVX-512) at load time, so it still goes fast on modern hardware while running cleanly on the pre-Haswell silicon. Use `lance.simd_info()` from Python to verify which tier was selected.
Same Python API (`import lancedb` works as usual). The compat wheel is compiled at the `x86-64-v2` baseline (Nehalem-class) and uses runtime SIMD dispatch in the embedded lance crate to pick the right kernel tier (scalar / AVX / AVX+FMA / AVX2+FMA / AVX-512) at load time, so it still goes fast on modern hardware while running cleanly on the pre-Haswell silicon (if `pylance` is installed, `lance.simd_info()` reports which tier was selected).
`lancedb` and `lancedb-compat` install to the same `lancedb/` namespace and conflict at install time. Pick one. To switch, `pip uninstall lancedb` first, then `pip install lancedb-compat` (or vice-versa).
@@ -40,8 +40,10 @@ Stable releases are created about every 2 weeks. For the latest features and bug
pip install --pre --extra-index-url https://pypi.fury.io/lancedb/ lancedb
```
If you also need the optional `lance` Python API, install the `pylance` extra
and include the Lance preview index:
Most applications only need `lancedb`. Install the optional `pylance` extra and
include the Lance preview index only when building query engines or connectors,
calling format-level `lance` APIs directly, or converting plain-scan lazy or
legacy blob columns with `to_pandas()`:
```bash
pip install --pre \