lancedb now holds a page at every path the site publishes, including the twelve
Enterprise ones. Those are real open-source pages -- what the capability is, and
what the embedded form does instead -- rather than blank placeholders, so the
open-source site is complete on its own. sophon becomes an overlay whose page
replaces the one at the same path, which means navigation lives in one place and
the two repositories only have to agree on a path.
That removes the whole class of problem where a page could only be reached from
one of the two trees. The open-source site went from twenty-four dangling
Enterprise links to one, and that one points at the dataset pages the build
repository owns.
Ports four changes that landed on the docs repository after this branch left it:
#344 indexing and quantization rewritten
#354 LeRobot native reader
#361 training/index renamed to training/data-loading and rewritten
#362 distance metric mismatch falls back to a brute-force scan
None applied as patches, because these files were merged or split here --
quantization into the vector index page, multi-vector into vector search,
performance into four pages. Each was ported by taking the upstream page and
re-applying the restructure on top, which also picked up a correction that would
otherwise have been lost: the stricter brute-force threshold for multi-vector
columns is a Cloud limit, not an Enterprise one.
The upstream LeRobot page linked to a Geneva page that no longer exists; it now
points at the feature-engineering pages that replaced it.
Six groups were verbs -- Manage, Ingest, Train, Operate, Secure, Deploy -- and
now name the thing rather than the action. Four more were sentence case.
The page labels that were verb phrases follow: Add Computed Column becomes
Computed Columns, Refresh Column becomes Column Refresh, Configuring Storage
becomes Configuration, and Explain becomes Query Plans, which is also more
accurate now that the page covers analyze as well.
SQL statement pages take the keyword -- SELECT, DDL, DML, REFRESH, SHOW, EXPLAIN
-- because a statement name is a noun and lowercase Select and Show read as
instructions. Concept pages there keep noun phrases: Joins, Window Functions,
Subqueries, Operators.
Two duplicate labels are resolved on the way. Reranking sat in Semantic Search
labelled Overview despite being the only page on its subject, and the Enterprise
client page sat in SQL Reference labelled SQL next to the reference Overview; it
is Client Connection now.
Title case leaves prepositions and articles lowercase, so Needle in a Haystack
and Hybrid Search on BEIR are already correct and unchanged.
Ten merges and renames, all in one direction: fewer pages, each covering a whole
subject.
Table absorbs schema evolution and versioning. Vector search absorbs
multi-vector. Explain absorbs explain analyze. Reranking was seven pages -- an
overview and six rerankers of a few dozen lines each -- and is now one. The
vector index page absorbs quantization, which was only ever a parameter of it.
Indexing moves to Manage, where the other catalog objects are. The operational
half stays in Operate as one Indexing page, absorbing GPU indexing -- which was
filed in sophon despite importing lancedb's snippets and being mostly
open-source content with one Enterprise-badged section.
Performance was a single page aggregating tips for four unrelated operations,
which is hard to keep current. Each section now sits with the operation it is
about: ingestion tips on Insert, index tips on Indexing, compaction tips on
Compaction, query tips on Search.
Data Types leaves Concepts. The type reference belongs in the SQL reference,
which already described the same types from the query side; Table now carries a
short summary that points there. The multimodal material it had absorbed is its
own page again, since storing blobs is a task rather than a type definition.
Query becomes Semantic Search, Branches becomes Branch, and the Enterprise
Security page becomes Compliance.
Redirects cover every moved URL, and 30 files were repointed at the merged
anchors.
The tab is Use Cases now, grouped General, Search, RAG, Feature Engineering,
Training, World Models -- one page per example, named for the example rather
than for its position in a tree.
The gallery pages that listed external notebooks become the More Examples entry
of the category they belong to, instead of standing in front of the real
tutorials. The top-level overview stops linking to itself through three levels
of index page and names the categories directly.
Branch experiments move to Feature Engineering, which is what they are an
experiment in. The agent plugin leads General, since it applies to everything
else in the tab.
LeRobotDataset moves to the Datasets tab, where the LeRobot datasets already
are, and Robotics moves to the top of that tab.
Feature engineering is now the overview plus one page per operation: add a
computed column, refresh a column, refresh a materialized view. Computed
columns covered declaring and filling on one page, which are separate decisions
with separate failure modes.
Functions move to Management, where the other catalog objects are, and absorb
the environment-mismatch guidance from the Enterprise page -- a symptom table
worth having beside the parameters that cause it.
The Enterprise feature-engineering page is merged into the overview. Two of its
claims were stale: create_materialized_view is implemented for db:// connections
in remote/db.py, so views are not local-only, and the tier is Enterprise rather
than 'Cloud and Enterprise'.
The REST reference leaves the sidebar. Mintlify only generates per-endpoint
pages from a navigation group -- hidden, dropdowns, global anchors and a
top-level openapi key were all tried -- so removing the group removes those
pages. The overview survives as an unlisted page, linked from SDKs & SQL, and
the spec still ships for anything that wants to read it directly.
The Documentation tab now reads as a manual over Python, TypeScript and SQL
rather than a Python guide with SQL bolted on.
Basics replaces the Catalog section: what the catalog holds, what each API
surface can do, jobs, agentic skills, and what Lance is. Management is new and
is the CRUD and listing reference for every catalog object, with a column per
language so the gaps are visible -- TypeScript has no function or SQL methods,
SQL needs a deployment.
Search becomes Query and gains a SELECT reference, since an analytical query is
a query whether it came from a builder or a statement.
Jobs is new. Every long-running operation already returned the same handle from
an _async method; nothing documented it.
The SQL function reference gains what Lance registers on top of DataFusion:
contains_tokens, backed by the FTS index, and the JSON accessors, backed by a
JSON scalar index. Geospatial st_* functions are behind a build feature LanceDB
does not ship, which is now stated rather than left to a confusing error.
Removes the Geneva pages' orphaned snippets, merges FTS Examples into Full-Text
Search, and splits Query Optimization into Explain and Explain Analyze.
The feature engineering documentation describes the geneva package, a separate
SDK. The capability it covers now lives in this client — materialized views,
computed columns, and registered functions are all shipped API with no pages
describing them.
Four pages cover the three mechanisms and, more usefully, when each applies.
The overview leads with that choice rather than with concepts, because the
mechanisms overlap and picking the wrong one is the expensive mistake.
Two things they say plainly that the API reference cannot:
Declaring and filling are separate everywhere, and filling never revisits a row
it has already done. That makes refreshes idempotent and appends cheap, and it
means a filled value does not track its inputs — surprising if you expect a
spreadsheet, so it is stated rather than left to be discovered.
The availability split runs the opposite way to expectation. Materialized views
are local only and raise on a remote connection; functions are Cloud and
Enterprise only and raise on a local one. Someone reading about both in one
sitting will otherwise assume the usual direction and be wrong in one of them.
Examples are the tested ones from the docstrings, so they cannot drift from the
behaviour they describe.
The training and tutorial pages that pointed into the geneva documentation now
point here.
The pages describing this code lived in lancedb/docs, a repository with ten
stars that neither contributing guide mentioned. A user who hit a wrong page
while reading the source had to find a separate, near-invisible repository to
fix it. These 102 pages now sit beside the code, and a documentation fix is a
pull request in the repository the reader already had open.
`docs/web` is a complete Mintlify site: `cd docs/web && mint dev` serves it with
no other checkout. Enterprise pages stay private and are merged in when the
published site is assembled, so links into them do not resolve here — every
open-source page does.
`docs/web-tests` holds the tests the code examples are extracted from. They ran
against a published wheel before; here they can run against this repository's
own SDK, so an example that stops compiling fails the pull request that broke
it rather than being noticed later.
Both directories sit beside `docs/src` and are invisible to it: mkdocs reads
`docs_dir: src`, so the SDK reference builds and deploys exactly as before.
Jack Ye