The tab now runs write, read, curate, derive, train before it reaches the
management side: Ingestion, Semantic Search, Data Curation, Feature Engineering,
Training, then Management and Operations. Indexing joins Operations, where the
index maintenance page already was, and gains it as Build & Repair.
Ingestion gains a Create Table page. It was a section inside the overview, so
the one thing a new reader needs first was the one thing the sidebar did not
show.
Data Curation is new: Aggregate, Sampling, Join, Windowing. The SQL reference
answers what the syntax is; these answer why you would reach for it, framed as
the questions curation actually asks -- what do I have, what can I afford, what
else do I know, which one do I keep. Each links to the reference for the
complete grammar rather than repeating it.
They carry the things a reference page would not: that count(*) minus count(col)
is your coverage, that sampling both sides of a join gives a percent rather than
ten, that a window ordered by a value you recompute moves rows between train and
test, and that a one percent sample tells you about the head and nothing about
the tail.
Six groups were verbs -- Manage, Ingest, Train, Operate, Secure, Deploy -- and
now name the thing rather than the action. Four more were sentence case.
The page labels that were verb phrases follow: Add Computed Column becomes
Computed Columns, Refresh Column becomes Column Refresh, Configuring Storage
becomes Configuration, and Explain becomes Query Plans, which is also more
accurate now that the page covers analyze as well.
SQL statement pages take the keyword -- SELECT, DDL, DML, REFRESH, SHOW, EXPLAIN
-- because a statement name is a noun and lowercase Select and Show read as
instructions. Concept pages there keep noun phrases: Joins, Window Functions,
Subqueries, Operators.
Two duplicate labels are resolved on the way. Reranking sat in Semantic Search
labelled Overview despite being the only page on its subject, and the Enterprise
client page sat in SQL Reference labelled SQL next to the reference Overview; it
is Client Connection now.
Title case leaves prepositions and articles lowercase, so Needle in a Haystack
and Hybrid Search on BEIR are already correct and unchanged.
Ten merges and renames, all in one direction: fewer pages, each covering a whole
subject.
Table absorbs schema evolution and versioning. Vector search absorbs
multi-vector. Explain absorbs explain analyze. Reranking was seven pages -- an
overview and six rerankers of a few dozen lines each -- and is now one. The
vector index page absorbs quantization, which was only ever a parameter of it.
Indexing moves to Manage, where the other catalog objects are. The operational
half stays in Operate as one Indexing page, absorbing GPU indexing -- which was
filed in sophon despite importing lancedb's snippets and being mostly
open-source content with one Enterprise-badged section.
Performance was a single page aggregating tips for four unrelated operations,
which is hard to keep current. Each section now sits with the operation it is
about: ingestion tips on Insert, index tips on Indexing, compaction tips on
Compaction, query tips on Search.
Data Types leaves Concepts. The type reference belongs in the SQL reference,
which already described the same types from the query side; Table now carries a
short summary that points there. The multimodal material it had absorbed is its
own page again, since storing blobs is a task rather than a type definition.
Query becomes Semantic Search, Branches becomes Branch, and the Enterprise
Security page becomes Compliance.
Redirects cover every moved URL, and 30 files were repointed at the merged
anchors.
The function catalogue is now real rather than a pointer. Scalar functions are
listed by taxonomy -- math, string, unicode, regex, date/time, array, map,
struct, crypto, type -- and the names come from what datafusion-functions 54
actually registers, not from what DataFusion documents on main. Aggregate and
window pages gain the same treatment.
Two things the audit corrected rather than copied.
SHOW TABLES does not work. DataFusion implements it over information_schema,
which this session does not enable, so it fails at plan time. It was documented
as working here, on the SHOW page and in three Manage tables; all four are
fixed, and the SQL overview now lists the absence alongside SHOW COLUMNS.
Prepared statements are a Flight SQL protocol feature, not a PREPARE statement.
A script carrying PREPARE ... AS SELECT will not run, while the same intent
expressed through a client will. The new page says so rather than copying
DataFusion's syntax.
Every section of the Documentation tab that has a SQL counterpart now links to
it -- 27 of 76 non-SQL pages, each pointing at the specific statement rather
than at the reference index.
The pages describing this code lived in lancedb/docs, a repository with ten
stars that neither contributing guide mentioned. A user who hit a wrong page
while reading the source had to find a separate, near-invisible repository to
fix it. These 102 pages now sit beside the code, and a documentation fix is a
pull request in the repository the reader already had open.
`docs/web` is a complete Mintlify site: `cd docs/web && mint dev` serves it with
no other checkout. Enterprise pages stay private and are merged in when the
published site is assembled, so links into them do not resolve here — every
open-source page does.
`docs/web-tests` holds the tests the code examples are extracted from. They ran
against a published wheel before; here they can run against this repository's
own SDK, so an example that stops compiling fails the pull request that broke
it rather than being noticed later.
Both directories sit beside `docs/src` and are invisible to it: mkdocs reads
`docs_dir: src`, so the SDK reference builds and deploys exactly as before.
Jack Ye