From 94f366a11a368b0d8e1bf0f24313aef586a8fd43 Mon Sep 17 00:00:00 2001 From: Brendan Clement Date: Mon, 9 Nov 2026 09:51:48 -0800 Subject: [PATCH] docs: show the Python filtered refresh next to the SQL form (#4923) Adds Python where= examples next to the SQL ones. Also fixes two examples that fail: - Date filters now use date '2026-01-01'. A bare string fails on DATE and TIMESTAMP columns (see [Dates, Timestamps, Decimals](https://docs.lancedb.com/search/filtering#dates-timestamps-decimals)). - The first example reads results with table.search().to_arrow(), because table.to_arrow() raises on Enterprise. where= needs lancedb 0.42.0 or later. --- .../feature-engineering/refresh-column.mdx | 37 ++++++++++++++----- docs/web/sql/dml/refresh-column.mdx | 4 +- 2 files changed, 29 insertions(+), 12 deletions(-) diff --git a/docs/web/feature-engineering/refresh-column.mdx b/docs/web/feature-engineering/refresh-column.mdx index 8f5726307..ac4ff93fe 100644 --- a/docs/web/feature-engineering/refresh-column.mdx +++ b/docs/web/feature-engineering/refresh-column.mdx @@ -22,7 +22,7 @@ Computed columns require LanceDB Enterprise. job = table.refresh_column_async("doubled") job.wait() -table.to_arrow().sort_by("x").to_pandas() +table.search().to_arrow().sort_by("x").to_pandas() # x doubled # 0 1 2 # 1 2 4 @@ -77,13 +77,21 @@ catch-up works when it's re-enabled after being off. ## Refreshing part of a table {#partial} -Enterprise In SQL a refresh can be scoped to a -predicate, which is useful for filling a slice first and checking it before -committing to the whole table: +Enterprise A refresh can be scoped to a predicate, +with `where=` in Python or `WHERE` in SQL, which is useful for filling a slice +first and checking it before committing to the whole table: +```python Python icon="python" +job = table.refresh_column_async( + "embedding", + where="created_at > date '2026-01-01'", +) +job.wait() +``` + ```sql SQL icon="database" -REFRESH COLUMN embedding ON documents WHERE created_at > '2026-01-01' +REFRESH COLUMN embedding ON documents WHERE created_at > date '2026-01-01' ``` @@ -92,9 +100,17 @@ whose outputs are already current. Repeating the same predicate repeats those Function calls and their cost. To select only null outputs within a slice: +```python Python icon="python" +job = table.refresh_column_async( + "embedding", + where="created_at > date '2026-01-01' AND embedding IS NULL", +) +job.wait() +``` + ```sql SQL icon="database" REFRESH COLUMN embedding ON documents - WHERE created_at > '2026-01-01' AND embedding IS NULL + WHERE created_at > date '2026-01-01' AND embedding IS NULL ``` @@ -247,13 +263,14 @@ and `gpuCount` in the `refreshColumnAsync` options. The job APIs also take the row filter and failure handling of the SQL `REFRESH COLUMN ... WHERE ... WITH (...)` form, so a refresh can be narrowed -to new rows and told to skip rows whose Function call fails: +to new rows (see [what a filter recomputes](#partial)) and told to skip rows +whose Function call fails: ```python job = table.refresh_column_async( "embedding", - where="published_at > '2026-01-01'", + where="created_at > date '2026-01-01'", error_policy="skip", max_failed_fraction=0.01, row_timeout_seconds=30, @@ -262,7 +279,7 @@ job = table.refresh_column_async( ```typescript const job = await table.refreshColumnAsync("embedding", { - where: "published_at > '2026-01-01'", + where: "created_at > date '2026-01-01'", errorPolicy: "skip", maxFailedFraction: 0.01, rowTimeoutSeconds: 30, @@ -276,7 +293,7 @@ let _job = table .refresh_column_async_with_options( "embedding", RefreshColumnOptions::default() - .where_clause("published_at > '2026-01-01'") + .where_clause("created_at > date '2026-01-01'") .error_policy(FunctionErrorPolicy::Skip) .max_failed_fraction(0.01) .row_timeout_seconds(30.0), diff --git a/docs/web/sql/dml/refresh-column.mdx b/docs/web/sql/dml/refresh-column.mdx index 98a360366..fcb78bd35 100644 --- a/docs/web/sql/dml/refresh-column.mdx +++ b/docs/web/sql/dml/refresh-column.mdx @@ -24,7 +24,7 @@ REFRESH COLUMN ON [TABLE] ```sql SQL icon="database" REFRESH COLUMN doubled ON my_table -REFRESH COLUMN embedding ON documents WHERE created_at > '2026-01-01' +REFRESH COLUMN embedding ON documents WHERE created_at > date '2026-01-01' ``` @@ -39,7 +39,7 @@ calls and their cost. To select only null outputs within a slice: ```sql SQL icon="database" REFRESH COLUMN embedding ON documents - WHERE created_at > '2026-01-01' AND embedding IS NULL + WHERE created_at > date '2026-01-01' AND embedding IS NULL ```