mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-23 20:22:37 +00:00
feat: configure automatic refresh when declaring computed columns (#4607)
Add `auto_refresh_enabled` and `auto_refresh_interval_seconds` to the
Rust builder and Python sync/async add-column APIs for computed columns
and registered Function outputs.
Python users can configure refresh when declaring a column:
```python
table.add_columns(
computed={"total": "price * quantity"},
auto_refresh_enabled=True,
auto_refresh_interval_seconds=300,
)
```
Omit the interval to let the server determine when to refresh:
```python
table.add_columns(computed={"discounted_total": "total * 0.9"})
```
The async API accepts the same options:
```python
await async_table.add_columns(
computed={"total": "price * quantity"},
auto_refresh_enabled=True,
auto_refresh_interval_seconds=300,
)
```
For an existing registered `FunctionApplication`, the options also apply
to Function columns. Disabling automatic refresh still permits explicit
refresh:
```python
table.add_columns(
{"features": function_application},
auto_refresh_enabled=False,
auto_refresh_interval_seconds=300,
)
table.refresh_column("features")
```
Options apply to every output declared by a call. Automatic refresh is
enabled by default; when the interval is omitted, the server determines
when to refresh. An explicit interval sets a minimum delay after a
successful refresh. The interval does not force recomputation of
unchanged inputs. Invalid intervals and options on one-time transforms
are rejected before declaration.
Native declarations persist `computed_column.auto_refresh_enabled` and
an explicitly supplied `computed_column.auto_refresh_interval_seconds`
in each output field’s metadata in the same commit as the declaration.
An omitted interval is neither stored nor sent in remote requests.
Remote declarations forward the settings to Cloud/Enterprise servers.
Local tables store these settings; automatic execution requires a
supporting server with scheduling enabled.
This commit is contained in:
@@ -51,6 +51,43 @@ The column is committed empty and the expression is stored, not evaluated:
|
||||
declaring one on a billion-row table costs what declaring one on an empty table
|
||||
costs. [`REFRESH COLUMN`](/sql/dml/refresh) fills it.
|
||||
|
||||
{/*
|
||||
Draft for automatic computed-column refresh. Enable this section when the
|
||||
server feature is released.
|
||||
|
||||
### Automatic refresh options {#automatic-refresh-options}
|
||||
|
||||
```sql
|
||||
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
||||
WITH (auto_refresh_enabled = true, auto_refresh_interval_seconds = 300);
|
||||
|
||||
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = false;
|
||||
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_interval_seconds = 600;
|
||||
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = true;
|
||||
```
|
||||
|
||||
Automatic refresh is enabled by default. An explicitly configured interval is
|
||||
stored in each computed output's field metadata; when omitted, the server
|
||||
determines when to refresh. The interval must be a positive integer
|
||||
and is minimum pacing after a successful refresh of that output; unchanged
|
||||
inputs do not force work. A new column is immediately eligible for its first
|
||||
refresh. The server's separate automatic-refresh switch must also be enabled.
|
||||
|
||||
These options also work beside a Function binding's `version` and `branch`:
|
||||
|
||||
```sql
|
||||
ALTER TABLE t ADD COLUMN (label, score) AS classify(text)
|
||||
WITH (version = 'fv_1', auto_refresh_enabled = true,
|
||||
auto_refresh_interval_seconds = 300);
|
||||
ALTER TABLE t ALTER COLUMN score SET auto_refresh_enabled = false;
|
||||
```
|
||||
|
||||
Declaration settings initialize every output in the group; later SET commands
|
||||
change one output. Disabling automatic refresh prevents new submissions while
|
||||
allowing inflight jobs to finish and explicit REFRESH COLUMN to run. Column
|
||||
SUSPEND/RESUME syntax is not supported.
|
||||
*/}
|
||||
|
||||
{/*
|
||||
Draft for the Functions release.
|
||||
|
||||
@@ -81,12 +118,14 @@ ALTER TABLE t ADD COLUMN (label, score) AS classify(text) WITH (version = 'fv_1'
|
||||
|
||||
```sql
|
||||
ALTER TABLE t ADD COLUMN c AS f(x) FILTER (WHERE keep) WITH (version = 'fv_1')
|
||||
ALTER TABLE t ADD COLUMN c AS f(x) WITH (num_workers = '2')
|
||||
REFRESH COLUMN c ON t WITH (num_workers = 2)
|
||||
```
|
||||
|
||||
<Note>
|
||||
`WITH` on a plain computed column accepts only `version`. Resource options such
|
||||
as `num_workers` apply to function columns, where there are workers to size.
|
||||
Declaration `WITH` accepts `auto_refresh_enabled` and
|
||||
`auto_refresh_interval_seconds`. Function bindings also accept `version` and
|
||||
`branch`. Worker resource options such as `num_workers` belong on
|
||||
`REFRESH COLUMN ... WITH (...)`.
|
||||
</Note>
|
||||
*/}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user