lancedb now holds a page at every path the site publishes, including the twelve
Enterprise ones. Those are real open-source pages -- what the capability is, and
what the embedded form does instead -- rather than blank placeholders, so the
open-source site is complete on its own. sophon becomes an overlay whose page
replaces the one at the same path, which means navigation lives in one place and
the two repositories only have to agree on a path.
That removes the whole class of problem where a page could only be reached from
one of the two trees. The open-source site went from twenty-four dangling
Enterprise links to one, and that one points at the dataset pages the build
repository owns.
Ports four changes that landed on the docs repository after this branch left it:
#344 indexing and quantization rewritten
#354 LeRobot native reader
#361 training/index renamed to training/data-loading and rewritten
#362 distance metric mismatch falls back to a brute-force scan
None applied as patches, because these files were merged or split here --
quantization into the vector index page, multi-vector into vector search,
performance into four pages. Each was ported by taking the upstream page and
re-applying the restructure on top, which also picked up a correction that would
otherwise have been lost: the stricter brute-force threshold for multi-vector
columns is a Cloud limit, not an Enterprise one.
The upstream LeRobot page linked to a Geneva page that no longer exists; it now
points at the feature-engineering pages that replaced it.
The tab now runs write, read, curate, derive, train before it reaches the
management side: Ingestion, Semantic Search, Data Curation, Feature Engineering,
Training, then Management and Operations. Indexing joins Operations, where the
index maintenance page already was, and gains it as Build & Repair.
Ingestion gains a Create Table page. It was a section inside the overview, so
the one thing a new reader needs first was the one thing the sidebar did not
show.
Data Curation is new: Aggregate, Sampling, Join, Windowing. The SQL reference
answers what the syntax is; these answer why you would reach for it, framed as
the questions curation actually asks -- what do I have, what can I afford, what
else do I know, which one do I keep. Each links to the reference for the
complete grammar rather than repeating it.
They carry the things a reference page would not: that count(*) minus count(col)
is your coverage, that sampling both sides of a join gives a percent rather than
ten, that a window ordered by a value you recompute moves rows between train and
test, and that a one percent sample tells you about the head and nothing about
the tail.
Jack Ye