rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-10-13 07:12:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: frame feature engineering as a lakehouse capability

Browse Source 
Functions and materialized views are objects in the Lance Multimodal Lakehouse,
alongside tables and indexes — created, versioned and refreshed like anything
else in it. The pages introduced them as client methods, which undersold what
they are and left the capability sounding like a helper rather than part of the
storage layer.

The materialized view page also framed Enterprise availability around which
client you are holding. A reader does not care about that; they care whether
views exist where their data lives. They do, refreshed across the cluster, and
the page now says so and points at the Enterprise documentation rather than
implying the feature is missing.
This commit is contained in:
Jack Ye
2026-09-04 00:09:04 -07:00
parent 65ed41333b
commit 6e1473278d
2 changed files with 27 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/web/feature-engineering/index.mdx
+12 -7
View File
@@ -1,16 +1,21 @@
---
title: "Feature engineering"
title: "LanceDB Feature Engineering"
sidebarTitle: "Overview"
description: "Derive columns and tables from data already in LanceDB, without moving it somewhere else to compute."
description: "Derive columns and tables from data already in the lakehouse, without moving it somewhere else to compute."
icon: layer-group
keywords: ["feature engineering", "computed columns", "materialized views", "functions", "udf"]
keywords: ["feature engineering", "computed columns", "materialized views", "functions", "lakehouse"]
---
Feature engineering in LanceDB happens where the data already is. Rather than
exporting a table, computing somewhere else and writing the result back, you
declare what a column or a table should contain and let LanceDB fill it.
Feature engineering happens where the data already is. Rather than exporting a
table, computing somewhere else and writing the result back, you declare what a
column or a table should contain and let LanceDB fill it.
Three mechanisms cover different shapes of that problem. They compose — a
**Functions and materialized views are objects in the Lance Multimodal
Lakehouse**, alongside tables and indexes. They are created, versioned, and
refreshed like anything else in it — not a separate system bolted alongside, and
not a pipeline you assemble out of external orchestration.
Three mechanisms cover different shapes of the problem. They compose — a
materialized view can select computed columns, and a computed column can be
filled by a registered function — but each is worth understanding on its own.
docs/web/feature-engineering/materialized-views.mdx
+15 -16
View File
@@ -1,14 +1,14 @@
---
title: "Materialized views"
sidebarTitle: "Materialized views"
description: "Derive a table from a query over another table, and keep it current incrementally."
description: "A lakehouse object: a table defined by a query over another table, kept current incrementally."
icon: table
keywords: ["materialized view", "create_materialized_view", "refresh", "incremental"]
---
A materialized view is a table whose contents are defined by a query over
another table. The query is recorded in the view's schema; a refresh computes
the rows.
A materialized view is a lakehouse object: a table whose contents are defined by
a query over another table. The query is recorded in the view's schema; a
refresh computes the rows.
The result is an ordinary table. It can be queried, indexed, and searched, and
it appears in `table_names()` — nothing about reading it is special.
@@ -74,23 +74,22 @@ Concurrent refreshes of one view do not duplicate rows. If two plan the same
source rows, they conflict on commit and the loser raises rather than writing
them twice — so a retry loop around `refresh()` is safe.
## Local databases only {#availability}
## Creating one against a deployment {#availability}
<Warning>
**This** materialized view — the one created through `lancedb.connect(...)` — is
supported on local databases only. Calling `create_materialized_view` on a
remote `db://` connection raises `NotImplementedError`.
`create_materialized_view` on this client builds a view over a **local**
database. Called on a remote `db://` connection it raises
`NotImplementedError`.
Enterprise has its own materialized views, created through a different client
with the same method name. If you are working against an Enterprise deployment,
that is the one you want; see the Enterprise feature engineering documentation.
The two are not the same feature and the method name is unfortunately shared, so
it is worth being sure which client you are holding.
Views exist on Enterprise deployments too, where a refresh runs across the
cluster rather than in your process — but they are created through the
Enterprise feature engineering interface rather than this method. See the
Enterprise documentation if that is where your data lives.
</Warning>
Note the direction, which is the reverse of
[functions](/feature-engineering/functions): those run only against Cloud and
Enterprise and raise locally. Neither mechanism is available on both.
The direction is the reverse of [functions](/feature-engineering/functions),
which register only against Cloud and Enterprise and raise locally. Worth
knowing before concluding either is unavailable.
## Working with the view {#working-with-the-view}