Commit Graph
8 Commits
Author SHA1 Message Date
Jack Ye 19f8f396ec docs: consolidate pages that were split for no reason
Ten merges and renames, all in one direction: fewer pages, each covering a whole
subject.

Table absorbs schema evolution and versioning. Vector search absorbs
multi-vector. Explain absorbs explain analyze. Reranking was seven pages -- an
overview and six rerankers of a few dozen lines each -- and is now one. The
vector index page absorbs quantization, which was only ever a parameter of it.

Indexing moves to Manage, where the other catalog objects are. The operational
half stays in Operate as one Indexing page, absorbing GPU indexing -- which was
filed in sophon despite importing lancedb's snippets and being mostly
open-source content with one Enterprise-badged section.

Performance was a single page aggregating tips for four unrelated operations,
which is hard to keep current. Each section now sits with the operation it is
about: ingestion tips on Insert, index tips on Indexing, compaction tips on
Compaction, query tips on Search.

Data Types leaves Concepts. The type reference belongs in the SQL reference,
which already described the same types from the query side; Table now carries a
short summary that points there. The multimodal material it had absorbed is its
own page again, since storing blobs is a task rather than a type definition.

Query becomes Semantic Search, Branches becomes Branch, and the Enterprise
Security page becomes Compliance.

Redirects cover every moved URL, and 30 files were repointed at the merged
anchors.
2026-09-14 00:57:26 -07:00
Jack Ye fe651ae003 docs: seed the SQL reference from DataFusion 54, and cross-link it
The function catalogue is now real rather than a pointer. Scalar functions are
listed by taxonomy -- math, string, unicode, regex, date/time, array, map,
struct, crypto, type -- and the names come from what datafusion-functions 54
actually registers, not from what DataFusion documents on main. Aggregate and
window pages gain the same treatment.

Two things the audit corrected rather than copied.

SHOW TABLES does not work. DataFusion implements it over information_schema,
which this session does not enable, so it fails at plan time. It was documented
as working here, on the SHOW page and in three Manage tables; all four are
fixed, and the SQL overview now lists the absence alongside SHOW COLUMNS.

Prepared statements are a Flight SQL protocol feature, not a PREPARE statement.
A script carrying PREPARE ... AS SELECT will not run, while the same intent
expressed through a client will. The new page says so rather than copying
DataFusion's syntax.

Every section of the Documentation tab that has a SQL counterpart now links to
it -- 27 of 76 non-SQL pages, each pointing at the specific statement rather
than at the reference index.
2026-09-14 00:42:31 -07:00
Jack Ye 139138c22e docs: move versioning and branching under Table, split schema from data evolution
Versioning and branching are properties of a table, not steps in getting data
in, so they join the Table group. Consistency stays with Ingestion, where the
question it answers -- when do other writers' rows become visible -- is asked.

The Ingesting Data page was in the Table group while an Ingestion overview
existed separately, so the two are merged. The large-dataset section was
duplicated between it and the insert page; the insert page keeps it.

Schema and Data Evolution covered two different things under one title.
Changing a table's shape stays in Table management; deriving values from data
already there becomes a Feature engineering page, next to computed columns and
functions, which are the same job at larger scale.
2026-09-09 23:23:14 -07:00
Jack Ye 8f37234a86 docs: split feature engineering, move functions to Management
Feature engineering is now the overview plus one page per operation: add a
computed column, refresh a column, refresh a materialized view. Computed
columns covered declaring and filling on one page, which are separate decisions
with separate failure modes.

Functions move to Management, where the other catalog objects are, and absorb
the environment-mismatch guidance from the Enterprise page -- a symptom table
worth having beside the parameters that cause it.

The Enterprise feature-engineering page is merged into the overview. Two of its
claims were stale: create_materialized_view is implemented for db:// connections
in remote/db.py, so views are not local-only, and the tier is Enterprise rather
than 'Cloud and Enterprise'.

The REST reference leaves the sidebar. Mintlify only generates per-endpoint
pages from a navigation group -- hidden, dropdowns, global anchors and a
top-level openapi key were all tried -- so removing the group removes those
pages. The overview survives as an unlisted page, linked from SDKs & SQL, and
the spec still ships for anything that wants to read it directly.
2026-09-09 23:19:06 -07:00
Jack Ye 4015560b83 docs: restructure around the three API surfaces
The Documentation tab now reads as a manual over Python, TypeScript and SQL
rather than a Python guide with SQL bolted on.

Basics replaces the Catalog section: what the catalog holds, what each API
surface can do, jobs, agentic skills, and what Lance is. Management is new and
is the CRUD and listing reference for every catalog object, with a column per
language so the gaps are visible -- TypeScript has no function or SQL methods,
SQL needs a deployment.

Search becomes Query and gains a SELECT reference, since an analytical query is
a query whether it came from a builder or a statement.

Jobs is new. Every long-running operation already returned the same handle from
an _async method; nothing documented it.

The SQL function reference gains what Lance registers on top of DataFusion:
contains_tokens, backed by the FTS index, and the JSON accessors, backed by a
JSON scalar index. Geospatial st_* functions are behind a build feature LanceDB
does not ship, which is now stated rather than left to a confusing error.

Removes the Geneva pages' orphaned snippets, merges FTS Examples into Full-Text
Search, and splits Query Optimization into Explain and Explain Analyze.
2026-09-09 22:20:19 -07:00
Jack Ye 6e1473278d docs: frame feature engineering as a lakehouse capability
Functions and materialized views are objects in the Lance Multimodal Lakehouse,
alongside tables and indexes — created, versioned and refreshed like anything
else in it. The pages introduced them as client methods, which undersold what
they are and left the capability sounding like a helper rather than part of the
storage layer.

The materialized view page also framed Enterprise availability around which
client you are holding. A reader does not care about that; they care whether
views exist where their data lives. They do, refreshed across the cluster, and
the page now says so and points at the Enterprise documentation rather than
implying the feature is missing.
2026-09-04 00:09:04 -07:00
Jack Ye 65ed41333b docs: distinguish this materialized view from the Enterprise one
Both the lancedb client and the geneva client expose
`create_materialized_view`, and they are not the same feature. The page said
materialized views are local only, which is true of this client and misleading
for anyone on Enterprise, where views are created through geneva and refreshed
on the cluster.

Found by assembling the combined site and reading the Geneva pages against the
new ones: they described the same call as distributed and Enterprise while this
one described it as raising. Both were accurate about their own client and
neither said which client it meant.
2026-09-04 00:06:54 -07:00
Jack Ye c0fe2eacda docs: document feature engineering against the LanceDB API
The feature engineering documentation describes the geneva package, a separate
SDK. The capability it covers now lives in this client — materialized views,
computed columns, and registered functions are all shipped API with no pages
describing them.

Four pages cover the three mechanisms and, more usefully, when each applies.
The overview leads with that choice rather than with concepts, because the
mechanisms overlap and picking the wrong one is the expensive mistake.

Two things they say plainly that the API reference cannot:

Declaring and filling are separate everywhere, and filling never revisits a row
it has already done. That makes refreshes idempotent and appends cheap, and it
means a filled value does not track its inputs — surprising if you expect a
spreadsheet, so it is stated rather than left to be discovered.

The availability split runs the opposite way to expectation. Materialized views
are local only and raise on a remote connection; functions are Cloud and
Enterprise only and raise on a local one. Someone reading about both in one
sitting will otherwise assume the usual direction and be wrong in one of them.

Examples are the tested ones from the docstrings, so they cannot drift from the
behaviour they describe.

The training and tutorial pages that pointed into the geneva documentation now
point here.
2026-09-03 23:39:51 -07:00