SQL now leads with why you'd reach for it beyond analytics: it's served over
Flight SQL, so it's the way in for any language without a native LanceDB SDK,
and broadly every public SDK method has a corresponding SQL statement. Closes
by pointing at the SQL Reference as the spec that defines the precise
behavior behind both the SQL surface and the SDK operations it mirrors.
Moving both into Getting Started (before Agentic Skills) means a reader knows
what the API surface even is before being told how to hand it to an agent.
FAQ content is redistributed into the pages it actually belongs to: license
and Parquet-comparison notes into the Lance format page, scale guidance into
Architecture, index-necessity and refine_factor guidance into the vector
index page, the multiprocessing/fork gotcha into SDKs and SQL, and the
weak-consistency recommendation into Consistency. General query-performance
issues are dropped since Explain and Explain Analyze already cover that
ground.
The new Support page under Deployment carries the OSS Discord and Enterprise
support-email channels that used to live in Troubleshooting.
OSS docs now own the full usage content for features that only run against
an Enterprise deployment, since the client interface itself lives here;
Enterprise docs keep only the parts specific to its own behavior. Sophon's
overlay for these five pages is dropped since the OSS versions are now the
complete reference.
SQL Reference is restructured to match an object-type SQL manual (DDL by
object, then DML, then DQL, then Expressions) instead of the old flat,
loosely-ordered page list, absorbing the SQL client-connection guide and
FTS-via-SQL examples into it.
One page was carrying two layers. The catalog answers what exists and what it is
called -- names, hierarchy, resolution, and the grants that follow the
hierarchy. The lakehouse is everything that acts on those names: storing rows,
searching them, deriving new ones, and keeping it all fast.
Separating them makes the dependency visible. The catalog is a specification
another engine can implement, which is why a Lance table is reachable without
LanceDB; the lakehouse is where the cost and the tuning live. Governance belongs
to the catalog because the hierarchy is what a grant attaches to, and
maintenance belongs to the lakehouse because it is the files being rewritten.
Also fixes a corruption from an earlier bulk link rewrite: "central
metadata/catalog services" in prose had become a path.
Query and Job were one page because they share the _async convention, but they
are different things: a query returns rows and streams them, a job returns an
outcome you wait for. Two pages now, query first, each carrying a short pointer
to the other rather than the shared framing.
Agentic Skills absorbs the agent plugin tutorial, which was a separate page in
Use Cases describing the same plugin in more detail -- the two had diverged, with
the tutorial carrying the fuller install instructions. It also moves to Getting
Started: installing the plugin is something you do before writing anything, not
a concept to understand.
Ten merges and renames, all in one direction: fewer pages, each covering a whole
subject.
Table absorbs schema evolution and versioning. Vector search absorbs
multi-vector. Explain absorbs explain analyze. Reranking was seven pages -- an
overview and six rerankers of a few dozen lines each -- and is now one. The
vector index page absorbs quantization, which was only ever a parameter of it.
Indexing moves to Manage, where the other catalog objects are. The operational
half stays in Operate as one Indexing page, absorbing GPU indexing -- which was
filed in sophon despite importing lancedb's snippets and being mostly
open-source content with one Enterprise-badged section.
Performance was a single page aggregating tips for four unrelated operations,
which is hard to keep current. Each section now sits with the operation it is
about: ingestion tips on Insert, index tips on Indexing, compaction tips on
Compaction, query tips on Search.
Data Types leaves Concepts. The type reference belongs in the SQL reference,
which already described the same types from the query side; Table now carries a
short summary that points there. The multimodal material it had absorbed is its
own page again, since storing blobs is a task rather than a type definition.
Query becomes Semantic Search, Branches becomes Branch, and the Enterprise
Security page becomes Compliance.
Redirects cover every moved URL, and 30 files were repointed at the merged
anchors.
The function catalogue is now real rather than a pointer. Scalar functions are
listed by taxonomy -- math, string, unicode, regex, date/time, array, map,
struct, crypto, type -- and the names come from what datafusion-functions 54
actually registers, not from what DataFusion documents on main. Aggregate and
window pages gain the same treatment.
Two things the audit corrected rather than copied.
SHOW TABLES does not work. DataFusion implements it over information_schema,
which this session does not enable, so it fails at plan time. It was documented
as working here, on the SHOW page and in three Manage tables; all four are
fixed, and the SQL overview now lists the absence alongside SHOW COLUMNS.
Prepared statements are a Flight SQL protocol feature, not a PREPARE statement.
A script carrying PREPARE ... AS SELECT will not run, while the same intent
expressed through a client will. The new page says so rather than copying
DataFusion's syntax.
Every section of the Documentation tab that has a SQL counterpart now links to
it -- 27 of 76 non-SQL pages, each pointing at the specific statement rather
than at the reference index.
Namespace and Table were the only objects behind a nested group. Both are now a
single page like Database, Materialized view, View and Function, so Manage is a
flat list of the things a catalog holds.
Namespace merges its two pages, which were a concept page and a usage page for
the same object.
Table is a new page. The old tables/index was a 672-line basic-usage tutorial
whose every section had acquired a better home during the restructure --
connecting, ingesting, searching, filtering, adding and dropping columns -- to
the point that its own Next steps cards pointed at the pages that superseded it.
It is replaced by a table object page: what a table carries, create, open, list,
rename, clone, drop, and links out.
Schema evolution, versioning and branching stay as siblings rather than folding
in. Merging them would have made one 1,800-line page, which is not what the
other object pages look like.
Every group now answers exactly one question. Get started and Concepts cover
what the system is; the verb groups cover doing something with it; Operate,
Secure and Deploy cover running it; SQL reference is lookup. Maintenance becomes
Operate, Governance becomes Secure, and the rest take verbs to match.
Three new pages in Concepts and Get started:
Guarantees states what a write promises when it returns, which version a reader
sees, what survives a failure, and -- the half usually missing -- what is not
guaranteed: no multi-table transactions, no read-your-writes across processes
without an interval, unenforced primary keys that do not enforce, no background
compaction.
Limits collects bounds that were stated in passing across a dozen pages, with
the numbers verified against the code rather than inferred: name lengths, job
event caps, SHOW ERRORS bounds, the sampling range that is refused rather than
clamped.
Architecture illustrates embedded against distributed as the one deployment
decision, and is written to cover both rather than only Enterprise.
Indexing splits by question. Choosing an index is a Query decision and moves
there; building, repairing and GPU-building it stay in Operate.
SQL becomes a top-level reference of fifteen pages instead of a sub-group of
Query. Auditing sophon-sql turned up a large undocumented surface: ALTER TABLE
ADD COLUMN in its plain, computed and function forms with FILTER, IGNORE NULLS
and OVER modifiers; SET UNENFORCED PRIMARY KEY; SET and UNSET WAL with sharding;
CREATE FUNCTION with its dependency options; REFRESH COLUMN and REFRESH
MATERIALIZED VIEW; and the SHOW family. None of it was written down.
SELECT and Enterprise SQL sat apart in Query while documenting the same engine.
They are now one SQL group: overview, Select, Aggregation, Join, Window,
Sampling, Functions -- one page per construct, with the Enterprise client-setup
and FTS pages merged in rather than parallel to them.
Sampling is new. TABLESAMPLE is implemented and was undocumented: BERNOULLI
picks rows, SYSTEM picks Lance fragments, REPEATABLE fixes the draw so a sample
works as a stable evaluation set. Its refusals are documented too -- a
percentage over 100, a fractional row count, BUCKET, the SAMPLE spelling and
the OFFSET modifier are all errors, because a sampling clause that were read and
not applied would be a wrong answer rather than a missing feature.
Schema & Data Types is one page again, with the multimodal content merged into
its blob section rather than sitting beside it.
Agentic Skills stops being a tab. Both its pages are tutorials, so they join the
Tutorials tab; the Basics page that introduces them stays.
Basics gains a Schema & Data Types section -- the Arrow types a table can hold,
the extension types Lance adds on top (lance.json, lance.blob.v2,
lance.bfloat16), and where geospatial actually stands. The multimodal page moves
in beside it. Consistency joins Basics too, after Query & Job.
Query & Job now leads with the thing the _async convention actually gives you: a
handle. execute_query_async returns a query, everything else returns a job, and
both answer the same four questions -- which one, how is it going, give me the
result, stop. Only the third differs, because a query produces rows to stream
and a job produces an outcome to wait for.
Management gains a Materialized View page. Creating one was documented only on
the refresh page, and the stable-row-ids requirement is a creation-time
decision that cannot be undone.
Embeddings move to Integrations as the Embedding Registry, which is what they
are -- a client-side feature that wraps provider models, sitting above the
providers it registers.
The demo gallery was three apps on one page; it is now three pages. Use Cases
folds into Tutorials as Tutorials and Use Cases.
Feature engineering is now the overview plus one page per operation: add a
computed column, refresh a column, refresh a materialized view. Computed
columns covered declaring and filling on one page, which are separate decisions
with separate failure modes.
Functions move to Management, where the other catalog objects are, and absorb
the environment-mismatch guidance from the Enterprise page -- a symptom table
worth having beside the parameters that cause it.
The Enterprise feature-engineering page is merged into the overview. Two of its
claims were stale: create_materialized_view is implemented for db:// connections
in remote/db.py, so views are not local-only, and the tier is Enterprise rather
than 'Cloud and Enterprise'.
The REST reference leaves the sidebar. Mintlify only generates per-endpoint
pages from a navigation group -- hidden, dropdowns, global anchors and a
top-level openapi key were all tried -- so removing the group removes those
pages. The overview survives as an unlisted page, linked from SDKs & SQL, and
the spec still ships for anything that wants to read it directly.
Basics now reads outward from the format: Lance Format, Lakehouse & Catalog,
Query & Job, SDKs & SQL, Agentic Skills.
Jobs becomes Query & Job. Async means two unrelated things here -- whether your
thread blocks, and whether the work outlives the call -- and nothing said so.
The API Reference tab is gone. Its SDK index is folded into SDKs & SQL, where
the surfaces are already compared; the generated REST pages move into Basics
beside it.
Connecting was described in passing on the catalog page. It is now a Management
page covering the URI forms and what each changes, deployment options, storage
options, and read consistency, since which one you pass decides what exists.
Quickstart ran out at multimodal storage. It now continues through a computed
column and its refresh into a PyTorch loader, which is the arc the rest of the
docs assume.
The Documentation tab now reads as a manual over Python, TypeScript and SQL
rather than a Python guide with SQL bolted on.
Basics replaces the Catalog section: what the catalog holds, what each API
surface can do, jobs, agentic skills, and what Lance is. Management is new and
is the CRUD and listing reference for every catalog object, with a column per
language so the gaps are visible -- TypeScript has no function or SQL methods,
SQL needs a deployment.
Search becomes Query and gains a SELECT reference, since an analytical query is
a query whether it came from a builder or a statement.
Jobs is new. Every long-running operation already returned the same handle from
an _async method; nothing documented it.
The SQL function reference gains what Lance registers on top of DataFusion:
contains_tokens, backed by the FTS index, and the JSON accessors, backed by a
JSON scalar index. Geospatial st_* functions are behind a build feature LanceDB
does not ship, which is now stated rather than left to a confusing error.
Removes the Geneva pages' orphaned snippets, merges FTS Examples into Full-Text
Search, and splits Query Optimization into Explain and Explain Analyze.
Jack Ye