Files
lancedb/docs/web/management/namespace.mdx
Jack YeandBrendan Clement 47b7babc08 docs: bring the open-source documentation into this repository (#4122)
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).

This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.

Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.

**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.

**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.

Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.

**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.

The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.

---------

Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
2026-10-07 11:15:00 -07:00

364 lines
15 KiB
Plaintext

---
title: "Namespaces"
sidebarTitle: "Namespace"
description: "Grouping tables into a hierarchy, and addressing them through it."
icon: folder-tree
keywords: ["namespace", "catalog", "hierarchy", "directory namespace", "rest namespace"]
---
import {
PyNamespaceTableOps,
PyNamespaceAdminOps,
TsNamespaceTableOps,
TsNamespaceAdminOps,
RsNamespaceAdminOps,
RsNamespaceTableOps,
} from '/snippets/connection.mdx';
Despite its name, LanceDB is not a "database" in the traditional sense -- it is a **Multimodal Lakehouse** that builds on the table abstraction,
similar to many other lakehouse projects. LanceDB exposes a catalog-level abstraction over the Lance table format, via a *namespace spec*.
If you're coming from traditional databases or lakehouses, you can think of a namespace as the catalog path that says where a table name lives.
Lance provides the **file** and **table** formats to store and manage your data and indexes.
LanceDB operates at the catalog layer (used to organize, discover, and operate on many Lance tables)
and provides a compute engine on top of the Lance format.
This is why many SDK methods in LanceDB, like `create_table`, `open_table`, `drop_table`, and
`rename_table`, accept namespace input. The SDK methods expose that input in the idiom of each
language: Python uses `namespace_path`, Rust uses builder methods like `.namespace(...)`, and
TypeScript uses `namespacePath` arguments.
<Note>
In SQL a namespaced table is addressed as a [dotted path](/sql/dql/select#namespaces).
</Note>
## Namespace hierarchy {#namespace-hierarchy}
Namespaces are generalizations of catalog specs that give platform developers a clean way to present Lance tables in the structures users expect. The diagram below shows how the hierarchy can go beyond a single level.
A namespace can contain a collection of tables, and it can also contain namespaces recursively.
![](/static/assets/images/namespaces/lance-namespace.png)
Before diving into examples, it helps to keep two terms in mind: the **namespace client** is the abstraction that presents a consistent namespace API, while the **namespace implementation** is the concrete backend that resolves namespaces and table locations (for example, a local directory or an external catalog).
If you want to go deeper, see the Lance format [namespace documentation](https://lance.org/format/namespace/).
## Namespace paths and names {#namespace-paths-and-names}
A namespace path is a list of components. For example, `["prod", "search"]` means the `search`
namespace inside the `prod` namespace. The empty path, `[]`, means the root namespace.
Each component is a name, not a filesystem path segment. Namespace names can't be empty, and each
component can contain only letters, numbers, underscores, hyphens, and periods. That keeps the same
identifier usable across local directory namespaces and REST namespace identifiers.
## Directory namespaces {#directory-namespaces}
The simplest namespace model in LanceDB is a single root namespace, often represented by one
directory:
```bash
./local_lancedb (root)
└── prod
└── search
└── user (table)
└── data (table)
```
As a user of LanceDB OSS, you might never notice namespaces at first, because LanceDB exposes the single-level hierarchy shown above, with the data stored in the `data/` directory, where the root namespace is implicit. Connecting to this namespace is as simple as connecting to the catalog root:
<CodeGroup>
```python Python icon="python"
import lancedb
# Connect to the directory namespace root
db = lancedb.connect("./local_lancedb")
```
```typescript TypeScript icon="square-js"
import * as lancedb from "@lancedb/lancedb";
// Connect to the directory namespace root
const db = await lancedb.connect("./local_lancedb");
```
```rust Rust icon="rust"
use lancedb::connect;
// Connect to the directory namespace root
let db = connect("./local_lancedb").execute().await?;
```
</CodeGroup>
This creates the default namespace directory (`data/`) under the specified root path.
You can also explicitly connect to a namespace using `lancedb.connect_namespace(...)` with the directory namespace implementation:
<CodeGroup>
```python Python icon="python"
import lancedb
# Local namespace-backed catalog root (DirectoryNamespace)
# See https://lance.org/format/namespace/dir/catalog-spec/
db = lancedb.connect_namespace("dir", {"root": "./local_lancedb"})
table_name = "user"
data = [{"id": 1, "vector": [0.1, 0.2], "name": "alice"}]
table = db.create_table(table_name, data=data, mode="create")
print(f"Created table: {table.name}")
# Created table: user
```
```typescript TypeScript icon="square-js"
import * as lancedb from "@lancedb/lancedb";
// Local namespace-backed catalog root (DirectoryNamespace)
// See https://lance.org/format/namespace/dir/catalog-spec/
const db = await lancedb.connectNamespace("dir", { root: "./local_lancedb" });
const table = await db.createTable(
"user",
[{ id: 1, vector: [0.1, 0.2], name: "alice" }],
{ mode: "create" },
);
console.log(`Created table: ${table.name}`);
// Created table: user
```
```rust Rust icon="rust"
use std::collections::HashMap;
use std::sync::Arc;
use arrow_schema::{DataType, Field, Schema};
// Local namespace-backed catalog root (DirectoryNamespace)
// See https://lance.org/format/namespace/dir/catalog-spec/
let mut properties = HashMap::new();
properties.insert("root".to_string(), "./local_lancedb".to_string());
let db = lancedb::connect_namespace("dir", properties)
.execute()
.await?;
let schema = Arc::new(Schema::new(vec![
Field::new("id", DataType::Int64, false),
]));
let table = db
.create_empty_table("user", schema)
.execute()
.await?;
println!("Created table: {}", table.name());
// Created table: user
```
</CodeGroup>
<Info>
- For simple use cases in LanceDB OSS, you don't need to go too deep into namespaces.
- To integrate LanceDB with external catalogs and to use it as a true **multimodal lakehouse**, it's useful to understand the different namespace implementations and how to use them in your organization's setup.
</Info>
## Remote or external catalog namespaces {#remote-or-external-catalog-namespaces}
The example above showed local directory-based namespaces. LanceDB also supports namespaces backed by remote object stores and external catalogs, via the REST namespace implementation.
For remote object stores with central metadata or catalog services (either commercial or open source),
use the REST namespace implementation. It is backed by REST routes
(for example `POST /v1/namespace/{id}/create` and `GET /v1/namespace/{id}/list`) and server-provided table locations.
For authentication, any property prefixed with `headers` is forwarded as an HTTP header
(for example `headers.Authorization` becomes `Authorization`, and `headers.X-API-Key` becomes `X-API-Key`).
LanceDB Enterprise REST requests use the `x-api-key` header for API-key authentication. Deployments
that route multiple databases through the same endpoint can also use headers such as
`x-lancedb-database` or `x-lancedb-database-prefix` for database context.
<CodeGroup>
```python Python icon="python"
import os
import lancedb
# Remote namespace-backed catalog root (RestNamespace)
# See https://lance.org/format/namespace/rest/catalog-spec/
db = lancedb.connect_namespace(
"rest",
{
"uri": "https://<your_catalog>.internal.<your_org>.com",
"headers.x-api-key": os.environ["API_KEY"],
# or:
# "headers.Authorization": f"Bearer {os.environ['REST_AUTH_TOKEN']}",
},
)
```
```typescript TypeScript icon="square-js"
import * as lancedb from "@lancedb/lancedb";
// Remote namespace-backed catalog root (RestNamespace)
// See https://lance.org/format/namespace/rest/catalog-spec/
const db = await lancedb.connectNamespace("rest", {
uri: "https://<your_catalog>.internal.<your_org>.com",
headers: {
"x-api-key": process.env.API_KEY ?? "",
// or:
// Authorization: `Bearer ${process.env.REST_AUTH_TOKEN}`,
},
});
```
```rust Rust icon="rust"
use std::collections::HashMap;
// Remote namespace-backed catalog root (RestNamespace)
// See https://lance.org/format/namespace/rest/catalog-spec/
let mut properties = HashMap::new();
properties.insert(
"uri".to_string(),
"https://<your_catalog>.internal.<your_org>.com".to_string(),
);
properties.insert(
"headers.x-api-key".to_string(),
std::env::var("API_KEY")?,
);
// or:
// properties.insert(
// "headers.Authorization".to_string(),
// format!("Bearer {}", std::env::var("REST_AUTH_TOKEN")?),
// );
let db = lancedb::connect_namespace("rest", properties)
.execute()
.await?;
```
</CodeGroup>
[LanceDB Enterprise](/enterprise) operates a REST namespace server on top of the Lance format, so any REST client that can speak the REST namespace API
contract can be used to interact with it. For authentication examples in LanceDB Enterprise, visit
the [Namespaces in SDKs](#namespaces-in-lancedb-enterprise) page.
## Best practices {#best-practices}
Below, we list some best practices for working with namespaces:
- For simple use cases and single, stand-alone applications, the directory-based root namespace is sufficient and requires no special configuration.
- For remote storage locations, introduce explicit namespaces when multiple teams, environments, or domains share the same catalog.
- Treat namespace paths as stable identifiers (for example `"prod/search"`, `"staging/recs"`).
- For maintainability reasons, avoid hard-coding object-store table paths in application code -- instead, prefer catalog identifiers + namespaces.
## Table operations with namespace paths {#table-operations-with-namespace-paths}
Let's imagine a scenario where your table management needs have evolved, and you now have the following multi-level structure to organize your tables outside the root namespace.
```
./local_lancedb (root)
└── prod
└── search
└── user (table)
└── data (table)
└── recommendations
└── user (table)
└── data (data)
```
Below, we show how you would express table operations within that namespace. Each item in the namespace
list (`["prod", "search"]`) represents a level in the namespace hierarchy, and the table name is
specified when you create, open, list, or drop it.
The SDK methods expose the namespace path in the idiom of each language:
- Python: pass `namespace_path=["prod", "search"]` to table operations.
- Rust: call builder methods such as `.namespace(vec!["prod".to_string(), "search".to_string()])`.
- TypeScript: pass a `namespacePath` array, for example `await db.openTable("user", ["prod", "search"])`.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{PyNamespaceTableOps}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsNamespaceTableOps}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsNamespaceTableOps}
</CodeBlock>
</CodeGroup>
<Note>
Using namespaces is **optional** in LanceDB, and most basic use cases do not require to work with them.
An empty namespace (`[]`), which is the default, means "root namespace", and the data will be stored in
the `data/` directory under the specified root path.
</Note>
## Namespace management APIs {#namespace-management-apis}
You can open/create/drop tables inside a namespace path (like `["prod", "search"]`).
All three SDKs expose namespace lifecycle operations directly.
In Python, use `lancedb.connect_namespace(...)` when calling namespace lifecycle methods such as
`create_namespace`, `list_namespaces`, `describe_namespace`, and `drop_namespace`.
In TypeScript, use `lancedb.connectNamespace(...)` and call `createNamespace`, `listNamespaces`,
`describeNamespace`, and `dropNamespace` on the returned `Connection`.
In Rust, use `lancedb::connect_namespace(...)` and call `create_namespace`, `list_namespaces`,
and `drop_namespace`.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{PyNamespaceAdminOps}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsNamespaceAdminOps}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsNamespaceAdminOps}
</CodeBlock>
</CodeGroup>
Namespace creation and deletion have modes that control what happens when the target already exists,
doesn't exist, or contains data:
- Create mode: `create` fails if the namespace already exists, `exist_ok` keeps the existing namespace, and `overwrite` replaces it.
- Drop mode: `fail` reports an error when the namespace doesn't exist, and `skip` treats a missing namespace as a successful no-op.
- Drop behavior: `restrict` keeps non-empty namespaces from being dropped, and `cascade` drops child namespaces and tables first.
Namespace path components can't be empty. Each component can contain only letters, numbers,
underscores, hyphens, and periods.
Listing APIs return the immediate children of the requested namespace path. Use `limit` with the
returned `page_token` to page through large catalogs; pass an empty namespace path (`[]`) when you
want to list from the root namespace.
## Namespaces in LanceDB Enterprise {#namespaces-in-lancedb-enterprise}
In LanceDB Enterprise deployments, configure namespace-backed federated databases in a TOML file under your deployment's `config` directory.
LanceDB Enterprise supports both directory-based (`ns_impl = "dir"`) and REST-based (`ns_impl = "rest"`) namespace implementations.
The example below shows how to configure a directory-based namespace implementation in LanceDB Enterprise.
```toml
# Federated database configuration for DirectoryNamespace
# This example uses minio storage
[federated_dbs.federated_dir_test]
ns_impl = "dir"
root = "s3://<your-deployment>/<your-dir>"
"storage.region" = "us-east-1"
"storage.endpoint" = "http://localhost:9000"
"storage.access_key_id" = "minioadmin"
"storage.secret_access_key" = "minioadmin"
"storage.allow_http" = "true"
# Far future expiration (year 2100)
"storage.expires_at_millis" = "4102444800000"
```
The example above uses MinIO, but the same approach applies to other cloud object storage platforms based on your deployment.
For REST-based namespace servers, you can specify the namespace implementation as `"rest"` with forwarding prefixed headers
for authentication and context propagation.
```toml
[federated_dbs.federated_rest_test]
ns_impl = "rest"
uri = "http://<your_org>.internal.catalog.com"
forward_header_prefixes = ["X-forward"]
```
With `forward_header_prefixes = ["X-forward"]`, any incoming header starting with `X-forward` is forwarded to
`http://<your_org>.internal.catalog.com`. This is useful for auth propagation, for example sending
`X-forward-authorization: Bearer xxxx`.
For the LanceDB REST API itself, requests use `x-api-key` for API-key authentication. If your endpoint
serves more than one database, LanceDB can also use headers such as `x-lancedb-database` or
`x-lancedb-database-prefix` to route the request to the right database context.