mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-12 23:02:28 +00:00
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).
This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.
Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.
**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.
**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.
Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.
**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.
The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.
---------
Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
108 lines
4.9 KiB
Plaintext
108 lines
4.9 KiB
Plaintext
---
|
|
title: "Stable World Model"
|
|
sidebarTitle: "Stable World Model"
|
|
description: "Use Stable World Model with LanceDB-backed datasets for reproducible world model research, fast data loading, and compact storage."
|
|
---
|
|
|
|
import {
|
|
PyFrameworksStableWorldmodelCollectLance,
|
|
PyFrameworksStableWorldmodelConvert,
|
|
PyFrameworksStableWorldmodelEvaluate,
|
|
PyFrameworksStableWorldmodelLoadLance,
|
|
} from '/snippets/integrations.mdx';
|
|
|
|
[Stable World Model](https://github.com/galilai-group/stable-worldmodel) is a research platform for collecting data, training world models, and evaluating policies with model-predictive control across standardized environments.
|
|
|
|
The LanceDB integration is built into Stable World Model's data format registry. Lance is the default backend for collected datasets, so a path ending in `.lance` gives you an append-friendly LanceDB table with episode-contiguous rows and fast indexed reads.
|
|
|
|
Random access speed is the bottleneck for world model training, since the loop repeatedly samples temporal windows from high-dimensional observations, actions, and rewards. The faster those windows arrive, the more GPU time goes into training rather than waiting on the data loader.
|
|
|
|
## Install {#install}
|
|
|
|
```bash
|
|
pip install stable-worldmodel
|
|
```
|
|
|
|
Datasets and checkpoints are stored under `$STABLEWM_HOME`, which defaults to `~/.stable_worldmodel/`.
|
|
|
|
## Collect data into Lance {#collect-data-into-lance}
|
|
|
|
Stable World Model uses Lance by default when you collect to a `.lance` path.
|
|
Replace `your_expert_policy` with the expert or scripted policy you use to collect demonstrations.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyFrameworksStableWorldmodelCollectLance}
|
|
</CodeBlock>
|
|
|
|
Every writer accepts a `mode` argument such as `append`, `overwrite`, or `error`. The default is append, so re-running collection extends the existing dataset.
|
|
|
|
## Load a Lance dataset for training {#load-a-lance-dataset-for-training}
|
|
|
|
The dataset loader autodetects the Lance format from the path.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyFrameworksStableWorldmodelLoadLance}
|
|
</CodeBlock>
|
|
|
|
Your model code stays focused on the world model objective while LanceDB handles the storage layout and read path.
|
|
|
|
## Evaluate with model-predictive control {#evaluate-with-model-predictive-control}
|
|
|
|
After training a world model on the Lance-backed dataset, Stable World Model can evaluate it with planning solvers such as CEM.
|
|
Replace `world_model` with the trained model object from your training loop.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyFrameworksStableWorldmodelEvaluate}
|
|
</CodeBlock>
|
|
|
|
## Convert between formats {#convert-between-formats}
|
|
|
|
Stable World Model can convert between registered dataset formats. A common workflow is to collect in Lance for fast training reads, then export to the video layout for compact inspection artifacts.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyFrameworksStableWorldmodelConvert}
|
|
</CodeBlock>
|
|
|
|
## Throughput {#throughput}
|
|
|
|
The Stable World Model README reports the following PushT benchmark results from `scripts/benchmark/compare_h5_lance.py`:
|
|
|
|
| Format | Source | Cache | samples/s | ms/step |
|
|
|:---|:---|:---|---:|---:|
|
|
| HDF5 | local | no-cache | 1,416.1 | 45.2 |
|
|
| HDF5 | local | cached | 1,474.0 | 43.4 |
|
|
| LanceDB | local | no-cache | 4,814.8 | 13.3 |
|
|
| LanceDB | local | cached | 4,431.3 | 14.4 |
|
|
| Video | local | - | 1,330.6 | 48.1 |
|
|
| LanceDB | s3 | no-cache | 3,183.7 | 20.1 |
|
|
| LanceDB | s3 | cached | 3,253.2 | 19.7 |
|
|
| HDF5 | s3 | no-cache | 9.1 | 7,032.5 |
|
|
| HDF5 | s3 | cached | 756.5 | 84.6 |
|
|
|
|
In that benchmark, local LanceDB reached about **3.4x** the no-cache throughput of local HDF5, while S3-backed LanceDB reached about **350x** the no-cache throughput of S3-backed HDF5. Even with cache enabled, S3-backed LanceDB was about **4.3x** faster than S3-backed HDF5.
|
|
|
|
These numbers come from the Stable World Model project's own benchmark setup, so they're best read as a reproducible directional baseline that may shift across environments, models, and storage configurations.
|
|
|
|
## Storage {#storage}
|
|
|
|
The same README reports these local storage sizes for the benchmark dataset:
|
|
|
|
| Format | Local size |
|
|
|:---|---:|
|
|
| HDF5 | 43.12 GB |
|
|
| LanceDB | 13.31 GB |
|
|
| Video | 496.29 MB |
|
|
|
|
LanceDB used about **69% less local storage than HDF5** in the reported benchmark, while preserving a table interface built for fast training reads and append-heavy collection.
|
|
|
|
## More resources {#more-resources}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Stable World Model README" href="https://github.com/galilai-group/stable-worldmodel/blob/main/README.md">
|
|
Installation, quick start, supported formats, benchmarks, environments, solvers, and citation.
|
|
</Card>
|
|
<Card title="Stable World Model docs" href="https://galilai-group.github.io/stable-worldmodel/">
|
|
Full upstream documentation with tutorials, API references, and guides.
|
|
</Card>
|
|
</CardGroup>
|