mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-15 08:12:26 +00:00
Jack Ye
19f8f396ec
Ten merges and renames, all in one direction: fewer pages, each covering a whole subject. Table absorbs schema evolution and versioning. Vector search absorbs multi-vector. Explain absorbs explain analyze. Reranking was seven pages -- an overview and six rerankers of a few dozen lines each -- and is now one. The vector index page absorbs quantization, which was only ever a parameter of it. Indexing moves to Manage, where the other catalog objects are. The operational half stays in Operate as one Indexing page, absorbing GPU indexing -- which was filed in sophon despite importing lancedb's snippets and being mostly open-source content with one Enterprise-badged section. Performance was a single page aggregating tips for four unrelated operations, which is hard to keep current. Each section now sits with the operation it is about: ingestion tips on Insert, index tips on Indexing, compaction tips on Compaction, query tips on Search. Data Types leaves Concepts. The type reference belongs in the SQL reference, which already described the same types from the query side; Table now carries a short summary that points there. The multimodal material it had absorbed is its own page again, since storing blobs is a task rather than a type definition. Query becomes Semantic Search, Branches becomes Branch, and the Enterprise Security page becomes Compliance. Redirects cover every moved URL, and 30 files were repointed at the merged anchors.
79 lines
3.4 KiB
Plaintext
79 lines
3.4 KiB
Plaintext
---
|
|
title: "Lakehouse and catalog"
|
|
sidebarTitle: "Lakehouse & Catalog"
|
|
description: "What a multimodal lakehouse holds, and how the catalog organizes it."
|
|
icon: sitemap
|
|
keywords: ["lakehouse", "catalog", "database", "namespace", "table", "view", "function"]
|
|
---
|
|
|
|
LanceDB is a **multimodal lakehouse**: data sits in open [Lance](/basics/lance)
|
|
tables on object storage, and a catalog gives those tables names, structure, and
|
|
the operations that act on them.
|
|
|
|
The lakehouse part is why images, embeddings, and the columns you derive from
|
|
them live in one table rather than in a warehouse plus a vector store plus a
|
|
feature store. The catalog part is what this page is about.
|
|
|
|
## The objects {#objects}
|
|
|
|
| Object | What it is |
|
|
| --- | --- |
|
|
| [Database](/management/database) | The root. A connection opens exactly one. |
|
|
| [Namespace](/management/namespace) | A named grouping inside a database. Can nest. |
|
|
| [Table](/management/table) | Rows and columns, versioned. What most operations act on. |
|
|
| [Materialized view](/feature-engineering/materialized-views) | A table whose contents come from a query, computed and stored. |
|
|
| [View](/management/view) | A saved query. Computed when read, not stored. |
|
|
| [Function](/management/function) | Registered code, applied to produce column values. |
|
|
|
|
```
|
|
database
|
|
└── namespace
|
|
├── table
|
|
├── materialized view (a table, defined by a query)
|
|
├── view (a query, not stored)
|
|
└── function
|
|
```
|
|
|
|
A namespace can contain further namespaces, so the hierarchy goes as deep as you
|
|
need. A table is reached by its path through it.
|
|
|
|
This page is about what the objects *are*. For the operations on them --
|
|
connecting, creating, listing, dropping -- see [Management](/management).
|
|
|
|
## A table is more than rows {#tables}
|
|
|
|
A Lance table carries things a warehouse table does not, and most of the rest of
|
|
these docs is about one of them:
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| **Columns** | including vectors, blobs, and columns [computed](/feature-engineering/add-computed-column) from other columns |
|
|
| **Versions** | every write commits one, so reads can [travel back](/management/table#versioning) |
|
|
| **Indexes** | vector, full-text, and scalar, built over the columns |
|
|
| **Branches** | isolated lines of change you can [merge or discard](/tables/branching) |
|
|
|
|
That is why "add a feature column" is a normal operation here rather than a
|
|
migration: it writes one new column and leaves the expensive data alone.
|
|
|
|
## Views and materialized views are different objects {#views}
|
|
|
|
The names are close enough to mislead. Both are defined by a query over other
|
|
tables; the difference is when that query runs.
|
|
|
|
- A **materialized view** stores its rows. A refresh computes them, and reads are
|
|
reads of stored data -- fast, and as current as the last refresh.
|
|
- A **view** stores only the query. Every read runs it, so a view is never stale
|
|
and never faster than the query it wraps.
|
|
|
|
Reach for a materialized view when the query is expensive and the data changes
|
|
less often than it is read. Reach for a view when correctness matters more than
|
|
latency, or when the query is cheap.
|
|
|
|
## What is not a catalog object {#elsewhere}
|
|
|
|
**Indexes** belong to a table rather than standing alone, and creating one is a
|
|
performance decision -- see [Maintenance](/indexing).
|
|
|
|
**Jobs** are not stored data at all; they are handles to work the server is
|
|
doing. See [Query and job](/basics/jobs).
|