feat: group a materialized view by GROUP BY (#4222)

A view could only map source rows one to one, or one to many through a
FROM-position item, so nothing could see all the rows sharing a key.

A view query now takes `GROUP BY expr, ...` with aggregate projections,
planned and executed by DataFusion over the lance scan. A group spans
fragments, so a grouped view is recomputed in full whenever its source
changes; each row's provenance is its group's smallest source row id.
Such a query is stored as definition format 2, so a reader that predates
grouping reports the view as unrefreshable instead of failing to parse
it.


Stacked on #4190.
This commit is contained in:
Wyatt Alt
2026-09-22 08:03:27 -07:00
committed by GitHub
parent 518d7ff1dd
commit 254688df96
10 changed files with 656 additions and 78 deletions
+1 -1
View File
@@ -54,7 +54,7 @@ describe("materialized views", () => {
// A newer writer's layout is reported, never guessed at.
for (const newer of [
`{"format":2,"query":${JSON.stringify(query)}}`,
`{"format":3,"query":${JSON.stringify(query)}}`,
'{"kind":"select_v3","source_table":"people"}',
]) {
expect(() => read(newer)).toThrow(/cannot refresh/);
+7 -4
View File
@@ -7,14 +7,17 @@ import { Table } from "./table";
/** Schema metadata key holding a materialized view's definition. */
export const DEFINITION_META_KEY = "mv.definition";
/** The stored layout this version reads: `{"format": 1, "query": "<SQL>"}`. */
export const DEFINITION_FORMAT = 1;
/**
* The newest stored layout this version reads: `{"format": N, "query": "<SQL>"}`,
* format 2 being a query with `GROUP BY`.
*/
export const DEFINITION_FORMAT = 2;
/**
* The query that defines a materialized view, as stored:
* `SELECT columns FROM [ns.]table [, function(args) AS alias | , UNNEST(column) AS alias]
* [WHERE predicate] [LIMIT n]`. A Function in `FROM` position yields one row
* per element it returns.
* [WHERE predicate] [GROUP BY expr, ...] [LIMIT n]`. A Function in `FROM`
* position yields one row per element it returns.
*/
export interface MaterializedViewDefinition {
/** The defining query, in the canonical spelling the server stores. */