mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 23:32:27 +00:00
Brendan Clement
69f11c259d
Since lancedb#4180, create_materialized_view runs the initial refresh before it returns (with_no_data=True skips it), so the page's first manual refresh now writes 0 rows, not 1. The example now reads the populated view and refreshes after a source change, and its snippets come from docs/web-tests/py/test_materialized_view.py. A note scopes the behavior: 0.39 and earlier create the view empty. Pages that said a refresh computes the rows now say it brings them up to date, and the SQL note no longer says an incremental refresh covers only added rows.
108 lines
4.4 KiB
Plaintext
108 lines
4.4 KiB
Plaintext
---
|
|
title: "Lance Catalog"
|
|
sidebarTitle: "Lance Catalog"
|
|
description: "The naming layer: what objects exist, how they nest, and who may reach them."
|
|
icon: sitemap
|
|
keywords: ["catalog", "namespace", "hierarchy", "governance", "database", "table", "view", "function"]
|
|
---
|
|
|
|
A catalog answers one question: **what exists, and what is it called.** It holds
|
|
names, the hierarchy they sit in, and the rules about who may reach them. It
|
|
does not store rows, run queries, or build indexes -- that is the
|
|
[lakehouse](/basics/lakehouse) on top of it.
|
|
|
|
Keeping the two apart is what lets a Lance table be reached by something that is
|
|
not LanceDB. The catalog is a specification -- the
|
|
[Lance Namespace](/basics/lance#standard) -- so any engine that implements it
|
|
resolves the same names to the same tables.
|
|
|
|
## The objects {#objects}
|
|
|
|
| Object | What it is |
|
|
| --- | --- |
|
|
| [Database](/management/database) | The root. A connection opens exactly one. |
|
|
| [Namespace](/management/namespace) | A named grouping inside a database. Can nest. |
|
|
| [Table](/management/table) | Rows and columns, versioned. What most operations act on. |
|
|
| [Materialized view](/management/materialized-view) | A table whose contents come from a query, computed and stored. |
|
|
| [View](/management/view) | A saved query. Computed when read, not stored. |
|
|
| [Function](/management/function) | Registered code, applied to produce column values. |
|
|
|
|
```
|
|
database
|
|
└── namespace
|
|
├── table
|
|
├── materialized view (a table, defined by a query)
|
|
├── view (a query, not stored)
|
|
└── function
|
|
```
|
|
|
|
A namespace can contain further namespaces, so the hierarchy goes as deep as
|
|
your organisation needs. A table is reached by its path through it:
|
|
`prod.events.clicks`.
|
|
|
|
This page is about what the objects *are*. For the operations on them, see
|
|
[Management](/management).
|
|
|
|
## Resolution {#resolution}
|
|
|
|
A name is resolved by walking the path, and the walk is the point: reaching a
|
|
table means traversing the namespaces above it. That is what makes the hierarchy
|
|
more than decoration -- it is the unit of both organisation and access.
|
|
|
|
A flat set of tables needs none of this. Namespaces start earning their keep when
|
|
several teams, environments or domains share one catalog and you want
|
|
`prod.search.documents` and `staging.search.documents` to be unambiguous.
|
|
|
|
## Governance {#governance}
|
|
|
|
<Badge color="red">Enterprise</Badge>
|
|
|
|
Because the catalog knows the hierarchy, it is the layer that can enforce
|
|
access. A grant names **one privilege** on **one object** for **one subject**,
|
|
and it inherits down the path: reaching a table requires `USAGE` on the
|
|
namespace containing it and on the database containing that.
|
|
|
|
That is the practical reason to put a boundary in the hierarchy -- a namespace
|
|
per team gives you something to grant on. See
|
|
[Authorization](/governance/authorization).
|
|
|
|
Embedded LanceDB has no such layer. Access is whatever the object store enforces
|
|
on the bucket, which has no concept of a table.
|
|
|
|
## Views and materialized views are different objects {#views}
|
|
|
|
The names are close enough to mislead. Both are catalog objects defined by a
|
|
query over other tables; the difference is when that query runs.
|
|
|
|
- A **materialized view** stores its rows, and a refresh brings them up to
|
|
date. Reads are reads of stored data -- fast, and as current as the last
|
|
refresh.
|
|
- A **view** stores only the query. Every read runs it, so a view is never stale
|
|
and never faster than the query it wraps.
|
|
|
|
Reach for a materialized view when the query is expensive and the data changes
|
|
less often than it is read. Reach for a view when correctness matters more than
|
|
latency, or when the query is cheap.
|
|
|
|
## What the catalog does not hold {#elsewhere}
|
|
|
|
**Indexes** belong to a table rather than standing alone. Creating one is a
|
|
performance decision, not a structural one -- see [Indexing](/indexing).
|
|
|
|
**Jobs** are not stored data at all; they are handles to work in progress. See
|
|
[Jobs](/basics/jobs).
|
|
|
|
**Rows** are in the [Lance](/basics/lance) files the catalog points at. The
|
|
catalog knows a table exists and where it lives; it does not know what is in it.
|
|
|
|
## Next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Lance Lakehouse" icon="layer-group" href="/basics/lakehouse">
|
|
What operates on these objects: search, compute, maintenance.
|
|
</Card>
|
|
<Card title="Management" icon="folder-tree" href="/management">
|
|
Creating, listing and dropping each one.
|
|
</Card>
|
|
</CardGroup>
|