feat: configure automatic refresh when declaring computed columns (#4607)

Add `auto_refresh_enabled` and `auto_refresh_interval_seconds` to the
Rust builder and Python sync/async add-column APIs for computed columns
and registered Function outputs.

Python users can configure refresh when declaring a column:

```python
table.add_columns(
    computed={"total": "price * quantity"},
    auto_refresh_enabled=True,
    auto_refresh_interval_seconds=300,
)
```

Omit the interval to let the server determine when to refresh:

```python
table.add_columns(computed={"discounted_total": "total * 0.9"})
```

The async API accepts the same options:

```python
await async_table.add_columns(
    computed={"total": "price * quantity"},
    auto_refresh_enabled=True,
    auto_refresh_interval_seconds=300,
)
```

For an existing registered `FunctionApplication`, the options also apply
to Function columns. Disabling automatic refresh still permits explicit
refresh:

```python
table.add_columns(
    {"features": function_application},
    auto_refresh_enabled=False,
    auto_refresh_interval_seconds=300,
)
table.refresh_column("features")
```

Options apply to every output declared by a call. Automatic refresh is
enabled by default; when the interval is omitted, the server determines
when to refresh. An explicit interval sets a minimum delay after a
successful refresh. The interval does not force recomputation of
unchanged inputs. Invalid intervals and options on one-time transforms
are rejected before declaration.

Native declarations persist `computed_column.auto_refresh_enabled` and
an explicitly supplied `computed_column.auto_refresh_interval_seconds`
in each output field’s metadata in the same commit as the declaration.
An omitted interval is neither stored nor sent in remote requests.
Remote declarations forward the settings to Cloud/Enterprise servers.
Local tables store these settings; automatic execution requires a
supporting server with scheduling enabled.
This commit is contained in:
Jack Ye
2026-10-21 20:27:11 -07:00
committed by GitHub
parent d97df0c82b
commit a4544c5472
13 changed files with 573 additions and 47 deletions
+27 -2
View File
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright The LanceDB Authors
use lancedb::table::computed_columns::ComputedColumnOptions;
use std::{
collections::HashMap,
sync::{
@@ -1803,35 +1804,59 @@ impl Table {
})
}
#[pyo3(signature = (columns, auto_refresh_enabled=None, auto_refresh_interval_seconds=None))]
pub fn add_computed_columns(
self_: PyRef<'_, Self>,
columns: Vec<(String, String)>,
auto_refresh_enabled: Option<bool>,
auto_refresh_interval_seconds: Option<u64>,
) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.inner_ref()?.clone();
let options = ComputedColumnOptions {
auto_refresh_enabled: auto_refresh_enabled.unwrap_or(true),
auto_refresh_interval_seconds,
};
options.validate().infer_error()?;
future_into_py(self_.py(), async move {
let mut builder = inner.add_columns();
for (name, expression) in columns {
builder = builder.computed(name, expression);
}
let result = builder.execute().await.infer_error()?;
let result = builder
.computed_options(options)
.execute()
.await
.infer_error()?;
Ok(AddColumnsResult::from(result))
})
}
#[pyo3(signature = (application_json, output_name, auto_refresh_enabled=None, auto_refresh_interval_seconds=None))]
pub fn add_function_columns(
self_: PyRef<'_, Self>,
application_json: String,
output_name: Option<String>,
auto_refresh_enabled: Option<bool>,
auto_refresh_interval_seconds: Option<u64>,
) -> PyResult<Bound<'_, PyAny>> {
let application =
lancedb::function::FunctionApplication::from_json(&application_json).infer_error()?;
let inner = self_.inner_ref()?.clone();
let options = ComputedColumnOptions {
auto_refresh_enabled: auto_refresh_enabled.unwrap_or(true),
auto_refresh_interval_seconds,
};
options.validate().infer_error()?;
future_into_py(self_.py(), async move {
let builder = match output_name {
Some(name) => inner.add_columns().function_as(name, application),
None => inner.add_columns().function(application),
};
let result = builder.execute().await.infer_error()?;
let result = builder
.computed_options(options)
.execute()
.await
.infer_error()?;
Ok(AddColumnsResult::from(result))
})
}