From 8cc8acd30c16f0f1be215b3cf549e7727d5f33b9 Mon Sep 17 00:00:00 2001 From: Brendan Clement Date: Tue, 6 Oct 2026 22:07:19 -0700 Subject: [PATCH] docs: correct the cascade limit and remote stable-row-id guidance The limits page still listed REFRESH MATERIALIZED VIEW ... CASCADE as parsed but not implemented, while the refresh page says it depends on the deployment's version. The row now says the same and links there. The refresh page told deployment users to create a view's source with new_table_enable_stable_row_ids. A db:// connection never sends that option: RemoteOptions keeps only the Azure account name, and a remote create_table sends only its mode and data. The page now says stable row ids are set up on the deployment, which depending on its version may give new tables them, refuse a view created through SQL over a source without them, or convert the source when a view is created through its REST API, and to check with its administrator. The architecture, limits, table and database pages no longer say stable row ids can never be enabled after creation, which that REST conversion contradicts; they say the SDK can only give them to a table it creates. The local heading now names the removed stable-row-id requirement instead of promising that any table can be a source, and notes the write-ahead-log checks that remain. --- docs/web/architecture.mdx | 5 ++-- docs/web/basics/limits.mdx | 13 +++++---- .../materialized-views.mdx | 28 +++++++++++++------ docs/web/management/database.mdx | 9 +++--- docs/web/management/materialized-view.mdx | 8 +++--- docs/web/management/table.mdx | 6 ++-- 6 files changed, 42 insertions(+), 27 deletions(-) diff --git a/docs/web/architecture.mdx b/docs/web/architecture.mdx index 28fc9e1eb..4ee1bc0e7 100644 --- a/docs/web/architecture.mdx +++ b/docs/web/architecture.mdx @@ -128,8 +128,9 @@ Because the storage layer is identical, moving is a connection-string change plus a data migration, not a rewrite. -The one thing to decide early is **stable row ids**, which cannot be enabled -after a table exists and which a deployment may require on a +The one thing to decide early is **stable row ids**: the SDK can give them to a +table only when it creates one on a local path or an object store, and a +deployment may require them on a [materialized view](/management/materialized-view) source. See [Limits](/basics/limits#schema). diff --git a/docs/web/basics/limits.mdx b/docs/web/basics/limits.mdx index 4303dc08e..f135de8d4 100644 --- a/docs/web/basics/limits.mdx +++ b/docs/web/basics/limits.mdx @@ -55,10 +55,13 @@ See [Jobs](/basics/jobs). | `IVF_RQ` vector dimensions | must be divisible by 8 | | FTS `max_token_length` | 40 by default; longer tokens are dropped | -Stable row ids **cannot be enabled after a table exists** -- the one schema -decision here that is genuinely irreversible. A deployment may require them on a -[materialized view](/management/materialized-view) source; a local view does -not. +Through the SDK, a table gets **stable row ids** only at creation, from the +`new_table_enable_stable_row_ids` storage option on a local path or an object +store; no SDK call adds them later, so decide early. A local +[materialized view](/feature-engineering/materialized-views#stable-row-ids) does +not need them. On a deployment, whether a view's source needs them and how a +table gets them depend on the deployment; see +[Against a deployment](/feature-engineering/materialized-views#availability). See [Data types](/sql/data-types). @@ -73,7 +76,7 @@ See [Data types](/sql/data-types). | `TABLESAMPLE BUCKET` | not supported | | `TABLESAMPLE` on a WAL-backed table | not supported | | `CREATE INDEX` | one column, server-chosen name | -| `REFRESH MATERIALIZED VIEW ... CASCADE` | parsed, not yet implemented | +| `REFRESH MATERIALIZED VIEW ... CASCADE` | depends on the deployment's version; see [REFRESH](/sql/dml/refresh#refresh-materialized-view) | A sampling percentage over 100 is an error rather than a full scan, because a query asking for 150 percent has a mistake in it and reading it as 100 would diff --git a/docs/web/feature-engineering/materialized-views.mdx b/docs/web/feature-engineering/materialized-views.mdx index 74bf81356..7b00a6ce4 100644 --- a/docs/web/feature-engineering/materialized-views.mdx +++ b/docs/web/feature-engineering/materialized-views.mdx @@ -49,14 +49,17 @@ matching row, two in this example. Omitting it selects every source column, expanded against the schema as it was at creation time. `where` filters source rows; `limit` caps the view. -## Any local table can be the source {#stable-row-ids} +## Local sources no longer need stable row ids {#stable-row-ids} -Locally, the source needs no special storage option. A local refresh does not -track which source rows the view came from, so stable row ids -(`new_table_enable_stable_row_ids`) are not required: the example above creates -its source with the defaults. Earlier development builds and the published -0.39.0 required them, and a deployment may still require them; see -[Against a deployment](#availability). +A local refresh does not track which source rows the view came from, so a local +view's source does not need stable row ids (`new_table_enable_stable_row_ids`): +the example above creates its source with the defaults. Earlier development +builds and the published 0.39.0 required them, and a deployment may still +require them; see [Against a deployment](#availability). + +The source is still checked, when the view is created and on every refresh: one +with an LSM write spec, or with retained write-ahead-log rows not yet compacted, +is refused. ## Every local refresh rebuilds the view {#incremental} @@ -85,8 +88,15 @@ there. How a deployment refreshes a view -- rebuilding the whole view, as above, or reconciling what changed in the source -- depends on its LanceDB Enterprise version and configuration, and so do the options [`REFRESH MATERIALIZED VIEW`](/sql/dml/refresh#refresh-materialized-view) -accepts and whether the source table needs stable row ids. Where a deployment -requires them, create the source with `new_table_enable_stable_row_ids`. +accepts and whether the source table needs stable row ids. + +Stable row ids are set up on the deployment, not by the SDK: a `db://` +connection does not send `new_table_enable_stable_row_ids`, and no SDK call adds +them to an existing table. Depending on its version and configuration, a +deployment may give new tables stable row ids, may refuse a view created through +SQL -- as the SDK creates one -- over a source without them, or may convert an +existing source when a view is created through its REST API. Check with the +deployment's administrator what it requires before relying on a source. `SHOW MATERIALIZED VIEWS` lists them, which is cheaper than `list_materialized_views` -- that method finds views by reading every table's diff --git a/docs/web/management/database.mdx b/docs/web/management/database.mdx index fbb9a8ee1..1f77e5b10 100644 --- a/docs/web/management/database.mdx +++ b/docs/web/management/database.mdx @@ -106,10 +106,11 @@ const db = await lancedb.connect("s3://bucket/path", { -`new_table_enable_stable_row_ids` is worth setting deliberately: stable row ids -cannot be enabled after a table exists, and a deployment may require them on a -[materialized view](/feature-engineering/materialized-views#availability)'s -source. +`new_table_enable_stable_row_ids` is worth setting deliberately: it gives stable +row ids to the tables this connection creates, and no SDK call adds them to an +existing table. A `db://` connection does not send it; a deployment decides +stable row ids itself (see +[Against a deployment](/feature-engineering/materialized-views#availability)). See [Storage](/deployment/storage) for the full set, including per-cloud credentials. diff --git a/docs/web/management/materialized-view.mdx b/docs/web/management/materialized-view.mdx index 69828144a..cbdb703b1 100644 --- a/docs/web/management/materialized-view.mdx +++ b/docs/web/management/materialized-view.mdx @@ -40,10 +40,10 @@ const view = await db.createMaterializedView("recent_docs", "documents", { ``` -`source` names the table it derives from. Locally any table can be one, with no -special storage option; a deployment may require it to have stable row ids (see -[Against a deployment](#deployment)). `select`, `where` and `limit` record the -query -- there is no separate definition step. +`source` names the table it derives from. A local source does not need stable +row ids or any other storage option; a deployment may require stable row ids +(see [Against a deployment](#deployment)). `select`, `where` and `limit` record +the query -- there is no separate definition step. ## Opening, listing and dropping {#manage} diff --git a/docs/web/management/table.mdx b/docs/web/management/table.mdx index 89a50c74b..5cf72a468 100644 --- a/docs/web/management/table.mdx +++ b/docs/web/management/table.mdx @@ -156,9 +156,9 @@ frame, or Pydantic models. [Ingesting data](/ingestion) covers each, plus `exist_ok` and `mode="overwrite"` for re-runnable scripts. -An explicit schema is where nullability is decided, and where you opt into -**stable row ids** -- which cannot be enabled later, and which a deployment may -require on a [materialized view](/management/materialized-view) source. See +An explicit schema is where nullability is decided. Creation is also the only +point at which the SDK can give a table **stable row ids**, which a deployment +may require on a [materialized view](/management/materialized-view) source. See [Limits](/basics/limits#schema).