mirror of
https://github.com/lancedb/lancedb.git
synced 2026-09-30 08:55:37 +00:00
feat: group a materialized view by GROUP BY (#4222)
A view could only map source rows one to one, or one to many through a FROM-position item, so nothing could see all the rows sharing a key. A view query now takes `GROUP BY expr, ...` with aggregate projections, planned and executed by DataFusion over the lance scan. A group spans fragments, so a grouped view is recomputed in full whenever its source changes; each row's provenance is its group's smallest source row id. Such a query is stored as definition format 2, so a reader that predates grouping reports the view as unrefreshable instead of failing to parse it. Stacked on #4190.
This commit is contained in:
@@ -54,7 +54,7 @@ describe("materialized views", () => {
|
||||
|
||||
// A newer writer's layout is reported, never guessed at.
|
||||
for (const newer of [
|
||||
`{"format":2,"query":${JSON.stringify(query)}}`,
|
||||
`{"format":3,"query":${JSON.stringify(query)}}`,
|
||||
'{"kind":"select_v3","source_table":"people"}',
|
||||
]) {
|
||||
expect(() => read(newer)).toThrow(/cannot refresh/);
|
||||
|
||||
@@ -7,14 +7,17 @@ import { Table } from "./table";
|
||||
/** Schema metadata key holding a materialized view's definition. */
|
||||
export const DEFINITION_META_KEY = "mv.definition";
|
||||
|
||||
/** The stored layout this version reads: `{"format": 1, "query": "<SQL>"}`. */
|
||||
export const DEFINITION_FORMAT = 1;
|
||||
/**
|
||||
* The newest stored layout this version reads: `{"format": N, "query": "<SQL>"}`,
|
||||
* format 2 being a query with `GROUP BY`.
|
||||
*/
|
||||
export const DEFINITION_FORMAT = 2;
|
||||
|
||||
/**
|
||||
* The query that defines a materialized view, as stored:
|
||||
* `SELECT columns FROM [ns.]table [, function(args) AS alias | , UNNEST(column) AS alias]
|
||||
* [WHERE predicate] [LIMIT n]`. A Function in `FROM` position yields one row
|
||||
* per element it returns.
|
||||
* [WHERE predicate] [GROUP BY expr, ...] [LIMIT n]`. A Function in `FROM`
|
||||
* position yields one row per element it returns.
|
||||
*/
|
||||
export interface MaterializedViewDefinition {
|
||||
/** The defining query, in the canonical spelling the server stores. */
|
||||
|
||||
Reference in New Issue
Block a user