Main now rebuilds a materialized view completely on every refresh
(#4373): there is no incremental mode, no `full` argument and no
stable-row-ID requirement on the source. The example ran against the
SDK built from this branch (0.40.0b13) failed, because a refresh after
adding Bea writes both matching rows, not one.
The example now creates its source with the default options and expects
two rows written by the refresh, and its generated snippet is
regenerated. Separate tests cover `with_no_data` (one matching row on
the first refresh) and refreshing to an earlier source version. The
refresh page explains the full rebuild and keeps the #incremental and
#stable-row-ids anchors for the sections that replace them. Its version
note now says what the published 0.39.0 does differently, as observed
with that release.
The management, table, database, limits, architecture and data-types
pages no longer say a view's source needs stable row ids, and the
feature-engineering and tutorial summaries no longer call refreshes
incremental.
The opt-in Enterprise walkthrough overwrote the table `function_demo`,
registered the Function `double`, then dropped both. On this SDK
drop_function removes the named Function, not only the version this run
created, so a run against a shared database could replace or remove
existing objects.
The walkthrough snippets keep their names. The test registers a copy of
`double` under a name unique to the run, creates the table under such a
name, and drops only those two objects. The table is no longer created
with mode="overwrite", so even a name collision cannot replace an
existing table. The remote test has not been run.
The page said every Function output must be non-nullable. On this SDK
only a scalar output must be. A named-struct output may declare nullable
fields: whether a row has a result at all is the struct's own validity,
which Enterprise keeps in a separate internal column when it expands the
result into sibling columns.
test_function_output_nullability checks both rules against the SDK,
including that a struct declared as a single nullable field is refused.
Since lancedb#4180, create_materialized_view runs the initial refresh
before it returns (with_no_data=True skips it), so the page's first manual
refresh now writes 0 rows, not 1. The example now reads the populated view
and refreshes after a source change, and its snippets come from
docs/web-tests/py/test_materialized_view.py. A note scopes the behavior:
0.39 and earlier create the view empty.
Pages that said a refresh computes the rows now say it brings them up to
date, and the SQL note no longer says an incremental refresh covers only
added rows.
Porting the upstream changes replaced snippets/indexing.mdx with a hand
copy from lancedb/docs, which dropped six exports the vector-index page
still imports (`mint validate` warns on each) and added
PyQuantizationCustomParams, which had no source here.
The quantization test region is ported from lancedb/docs and the module
is regenerated, so every export comes from a test again. The regenerated
PyQuantizationCustomParams matches the copy byte for byte. Also fixes
"convetion" and "entires" on the page.
"explictly" and "Additionaly", in snippet sources and their generated
snippets. integrations.mdx is updated in place: regenerating it whole
would also drop PyRerankingWatsonxUsage, which has no source.
The guide applied the local decorated callable to `table["value"]`, which
raises TypeError: a table is not subscriptable, and add_columns takes an
application of a registered version, not the callable's result.
The guide now walks one example from definition to filled column: define
`double`, call it locally, connect to Enterprise, create a table whose
`value` column matches the parameter's type and nullability, register the
function, apply `version(value=col("value"))`, refresh, wait, and read the
values back. Its snippets come from docs/web-tests/py/test_function.py.
The local tests in that file check what `@udf` sends for registration, how
a registered version binds a column, and that a local connection refuses
both registration and Function columns. The Enterprise walkthrough runs
only when LANCEDB_URI, LANCEDB_API_KEY and LANCEDB_HOST_OVERRIDE are set.
The page also stops promising that a retry after a failed refresh costs
only the unfinished rows.
The Documentation tab now reads as a manual over Python, TypeScript and SQL
rather than a Python guide with SQL bolted on.
Basics replaces the Catalog section: what the catalog holds, what each API
surface can do, jobs, agentic skills, and what Lance is. Management is new and
is the CRUD and listing reference for every catalog object, with a column per
language so the gaps are visible -- TypeScript has no function or SQL methods,
SQL needs a deployment.
Search becomes Query and gains a SELECT reference, since an analytical query is
a query whether it came from a builder or a statement.
Jobs is new. Every long-running operation already returned the same handle from
an _async method; nothing documented it.
The SQL function reference gains what Lance registers on top of DataFusion:
contains_tokens, backed by the FTS index, and the JSON accessors, backed by a
JSON scalar index. Geospatial st_* functions are behind a build feature LanceDB
does not ship, which is now stated rather than left to a confusing error.
Removes the Geneva pages' orphaned snippets, merges FTS Examples into Full-Text
Search, and splits Query Optimization into Explain and Explain Analyze.
The pages describing this code lived in lancedb/docs, a repository with ten
stars that neither contributing guide mentioned. A user who hit a wrong page
while reading the source had to find a separate, near-invisible repository to
fix it. These 102 pages now sit beside the code, and a documentation fix is a
pull request in the repository the reader already had open.
`docs/web` is a complete Mintlify site: `cd docs/web && mint dev` serves it with
no other checkout. Enterprise pages stay private and are merged in when the
published site is assembled, so links into them do not resolve here — every
open-source page does.
`docs/web-tests` holds the tests the code examples are extracted from. They ran
against a published wheel before; here they can run against this repository's
own SDK, so an example that stops compiling fails the pull request that broke
it rather than being noticed later.
Both directories sit beside `docs/src` and are invisible to it: mkdocs reads
`docs_dir: src`, so the SDK reference builds and deploys exactly as before.
Brendan Clement
Jack Ye