diff --git a/docs/web/feature-engineering/index.mdx b/docs/web/feature-engineering/index.mdx index af1697bdc..00c4c0e5c 100644 --- a/docs/web/feature-engineering/index.mdx +++ b/docs/web/feature-engineering/index.mdx @@ -1,16 +1,21 @@ --- -title: "Feature engineering" +title: "LanceDB Feature Engineering" sidebarTitle: "Overview" -description: "Derive columns and tables from data already in LanceDB, without moving it somewhere else to compute." +description: "Derive columns and tables from data already in the lakehouse, without moving it somewhere else to compute." icon: layer-group -keywords: ["feature engineering", "computed columns", "materialized views", "functions", "udf"] +keywords: ["feature engineering", "computed columns", "materialized views", "functions", "lakehouse"] --- -Feature engineering in LanceDB happens where the data already is. Rather than -exporting a table, computing somewhere else and writing the result back, you -declare what a column or a table should contain and let LanceDB fill it. +Feature engineering happens where the data already is. Rather than exporting a +table, computing somewhere else and writing the result back, you declare what a +column or a table should contain and let LanceDB fill it. -Three mechanisms cover different shapes of that problem. They compose — a +**Functions and materialized views are objects in the Lance Multimodal +Lakehouse**, alongside tables and indexes. They are created, versioned, and +refreshed like anything else in it — not a separate system bolted alongside, and +not a pipeline you assemble out of external orchestration. + +Three mechanisms cover different shapes of the problem. They compose — a materialized view can select computed columns, and a computed column can be filled by a registered function — but each is worth understanding on its own. diff --git a/docs/web/feature-engineering/materialized-views.mdx b/docs/web/feature-engineering/materialized-views.mdx index 966bc052c..1b27d55b4 100644 --- a/docs/web/feature-engineering/materialized-views.mdx +++ b/docs/web/feature-engineering/materialized-views.mdx @@ -1,14 +1,14 @@ --- title: "Materialized views" sidebarTitle: "Materialized views" -description: "Derive a table from a query over another table, and keep it current incrementally." +description: "A lakehouse object: a table defined by a query over another table, kept current incrementally." icon: table keywords: ["materialized view", "create_materialized_view", "refresh", "incremental"] --- -A materialized view is a table whose contents are defined by a query over -another table. The query is recorded in the view's schema; a refresh computes -the rows. +A materialized view is a lakehouse object: a table whose contents are defined by +a query over another table. The query is recorded in the view's schema; a +refresh computes the rows. The result is an ordinary table. It can be queried, indexed, and searched, and it appears in `table_names()` — nothing about reading it is special. @@ -74,23 +74,22 @@ Concurrent refreshes of one view do not duplicate rows. If two plan the same source rows, they conflict on commit and the loser raises rather than writing them twice — so a retry loop around `refresh()` is safe. -## Local databases only {#availability} +## Creating one against a deployment {#availability} -**This** materialized view — the one created through `lancedb.connect(...)` — is -supported on local databases only. Calling `create_materialized_view` on a -remote `db://` connection raises `NotImplementedError`. +`create_materialized_view` on this client builds a view over a **local** +database. Called on a remote `db://` connection it raises +`NotImplementedError`. -Enterprise has its own materialized views, created through a different client -with the same method name. If you are working against an Enterprise deployment, -that is the one you want; see the Enterprise feature engineering documentation. -The two are not the same feature and the method name is unfortunately shared, so -it is worth being sure which client you are holding. +Views exist on Enterprise deployments too, where a refresh runs across the +cluster rather than in your process — but they are created through the +Enterprise feature engineering interface rather than this method. See the +Enterprise documentation if that is where your data lives. -Note the direction, which is the reverse of -[functions](/feature-engineering/functions): those run only against Cloud and -Enterprise and raise locally. Neither mechanism is available on both. +The direction is the reverse of [functions](/feature-engineering/functions), +which register only against Cloud and Enterprise and raise locally. Worth +knowing before concluding either is unavailable. ## Working with the view {#working-with-the-view}