mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-18 00:24:01 +00:00
A broad documentation pass across the areas LanceDB Enterprise and Embedded diverge on most: **Data Curation** -- GPU Function execution, Slurm scheduling (and what Slurm is, for readers new to it), blob late materialization as its own page, joins, aggregation/profiling, sampling, near-duplicate detection, chunking, distributed shuffle, and IVF-partition clustering. **Feature Engineering** -- computed columns and materialized views, restructured to lead with the Function-based path and show the SQL form side by side with Python throughout; a single Auto Refresh page instead of duplicating scheduling details on every refresh page; adaptive execution; checkpointing; GPU and Slurm placement for refresh workers; a Training Cache page for the PyTorch data loader's automatic cache benefit. **Security & Governance** -- an Audit Log reference (schema, access, grants), Secrets promoted to an ordinary catalog object with their own SQL DDL and a vending/rotation guide, and privilege-table fixes verified against the authorization source. **Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry moved into a real Deployment area; Automated Cleanup folded into Intelligent Auto-Maintenance (every other maintenance decision already lived there); RTE Compaction and Advanced Conflict Resolution documented in depth. **Landing page** -- the Enterprise feature comparison table reorganized (several redundant or stale rows removed, several new areas added), and the "why Enterprise" bullets replaced with ones tied to a concrete page (Performance, Scale, Fault tolerance, Multicloud deployment, Simplify operations) instead of vague promises. Plus a full pass addressing lancedb-gatekeeper's review: a real test-safety bug (a quickstart test could silently overwrite and delete a deployment's own `frames` table), a fabricated SQL statement with no backing implementation, an actual dedup-guarantee bug in `take_row_ids`, and dozens of documentation-accuracy fixes verified against source rather than taken on faith -- each either fixed, or left with an explicit note on why it needs a feature or deployment owner's sign-off instead of being guessed at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
158 lines
5.6 KiB
Plaintext
158 lines
5.6 KiB
Plaintext
---
|
|
title: "Management"
|
|
sidebarTitle: "Overview"
|
|
description: "Create, list, and drop the objects in a catalog, from Python, TypeScript, or SQL."
|
|
icon: folder-tree
|
|
keywords: ["ddl", "create", "drop", "list", "catalog", "management"]
|
|
---
|
|
|
|
Use the SDKs or SQL to manage [catalog objects](/basics/catalog), including
|
|
tables, namespaces and views. The tables below provide a quick reference to
|
|
the available methods; each object's guide covers its setup and behavior.
|
|
|
|
Some long-running operations also provide a submission method that returns a
|
|
[job](/basics/jobs). See the operation's guide for how to track completion.
|
|
|
|
## Management APIs {#surface}
|
|
|
|
<Note>
|
|
SQL is available through the Enterprise Flight SQL endpoint.
|
|
See [SQL](/basics/sql#sql-is-remote).
|
|
</Note>
|
|
|
|
### Namespaces {#namespaces}
|
|
|
|
| | Python | TypeScript | SQL |
|
|
| --- | --- | --- | --- |
|
|
| create | `create_namespace` | `createNamespace` | -- |
|
|
| list | `list_namespaces` | `listNamespaces` | -- |
|
|
| describe | `describe_namespace` | `describeNamespace` | -- |
|
|
| exists | `namespace_exists` | -- | -- |
|
|
| drop | `drop_namespace` | `dropNamespace` | -- |
|
|
|
|
See [Namespaces](/management/namespace).
|
|
|
|
### Tables {#tables}
|
|
|
|
| | Python | TypeScript | SQL |
|
|
| --- | --- | --- | --- |
|
|
| create | `create_table` | `createTable` | `CREATE TABLE` |
|
|
| create empty | `create_table(schema=)` | `createEmptyTable` | `CREATE TABLE` |
|
|
| open | `open_table` | `openTable` | -- |
|
|
| list | `table_names`, `list_tables` | `tableNames`, `listTables` | -- |
|
|
| exists | `table_exists` | -- | -- |
|
|
| rename | `rename_table` | `renameTable` | -- |
|
|
| clone | `clone_table` | `cloneTable` | -- |
|
|
| drop | `drop_table`, `drop_table_async` | `dropTable`, `dropTableAsync` | `DROP TABLE` |
|
|
| drop all | `drop_all_tables` | `dropAllTables` | -- |
|
|
|
|
See [Tables](/management/table) for creating one and shaping its schema.
|
|
|
|
### Materialized views {#materialized-views}
|
|
|
|
| | Python | TypeScript | SQL |
|
|
| --- | --- | --- | --- |
|
|
| create | `create_materialized_view` | `createMaterializedView` | `CREATE MATERIALIZED VIEW` |
|
|
| open | `open_materialized_view` | `openMaterializedView` | -- |
|
|
| list | `list_materialized_views` | `listMaterializedViews` | `SHOW MATERIALIZED VIEWS` |
|
|
| refresh | `refresh` | `refresh` | `REFRESH MATERIALIZED VIEW` |
|
|
| drop | `drop_table` | `dropTable` | `DROP TABLE` |
|
|
|
|
<Note>
|
|
Local `list_materialized_views` reads table schemas to find views. On
|
|
Enterprise, it requests the server's view listing.
|
|
</Note>
|
|
|
|
See [Materialized views](/management/materialized-view) for creating one, and
|
|
[Refresh a materialized view](/feature-engineering/refresh-materialized-view) for keeping
|
|
it current.
|
|
|
|
### Views {#views}
|
|
|
|
| | Python | TypeScript | SQL |
|
|
| --- | --- | --- | --- |
|
|
| create | `create_view` | `createView` | `CREATE VIEW` |
|
|
| list | `list_views` | `listViews` | -- |
|
|
| drop | `drop_view`, `drop_view_async` | `dropView`, `dropViewAsync` | `DROP VIEW` |
|
|
|
|
<Badge color="red">Enterprise</Badge> Views exist only on an Enterprise cluster that
|
|
enables them; the SDK methods send the view's SQL query to the deployment. See
|
|
[Views](/management/view).
|
|
|
|
### Functions {#functions}
|
|
|
|
| | Python | TypeScript | SQL |
|
|
| --- | --- | --- | --- |
|
|
| create | `create_function`, `create_function_async` | -- | `CREATE FUNCTION` |
|
|
| get | `get_function` | -- | `DESCRIBE FUNCTION` |
|
|
| list | `list_functions` | -- | `SHOW FUNCTIONS` |
|
|
| drop | `drop_function` | -- | `DROP FUNCTION` |
|
|
|
|
<Badge color="red">Enterprise</Badge> A function runs on the server, and only on
|
|
a deployment that enables functions, so there is no local form. The TypeScript
|
|
SDK has no function methods; use Python or SQL.
|
|
See [Functions](/management/function).
|
|
|
|
### Jobs {#jobs}
|
|
|
|
| | Python | TypeScript | SQL |
|
|
| --- | --- | --- | --- |
|
|
| list | `list_jobs` | `listJobs` | `SHOW JOBS` |
|
|
| open | `open_job` | `openJob` | -- |
|
|
| cancel | `cancel_job` | `cancelJob` | -- |
|
|
| errors | `job.events` | `job.events` | `SHOW ERRORS` |
|
|
|
|
See [Job Framework](/basics/jobs).
|
|
|
|
### Indexes {#indexes}
|
|
|
|
| | Python | TypeScript | SQL |
|
|
| --- | --- | --- | --- |
|
|
| create | `create_index`, `create_index_async` | `createIndex`, `createIndexAsync` | `CREATE INDEX` |
|
|
| list | `list_indices` | `listIndices` | -- |
|
|
| stats | `index_stats` | `indexStats` | -- |
|
|
| drop | `drop_index` | `dropIndex` | `DROP INDEX` |
|
|
|
|
<Note>
|
|
SQL `CREATE INDEX` covers one column and takes no user-chosen name: the server
|
|
names it. For a multi-column or named index, use the SDK.
|
|
</Note>
|
|
|
|
Indexes are a performance decision rather than a structural one, so they are
|
|
covered under [Maintenance](/indexing).
|
|
|
|
## Work within a namespace {#namespace-paths}
|
|
|
|
Most connection-level operations take a `namespace_path` (`namespacePath`) to
|
|
act inside a namespace rather than at the database root:
|
|
|
|
<CodeGroup>
|
|
```python Python icon="python"
|
|
db.table_names(namespace_path=["prod", "events"])
|
|
tbl = db.open_table("clicks", namespace_path=["prod", "events"])
|
|
```
|
|
|
|
```typescript TypeScript icon="square-js"
|
|
await db.tableNames(["prod", "events"]);
|
|
const tbl = await db.openTable("clicks", ["prod", "events"]);
|
|
```
|
|
|
|
```sql SQL
|
|
SELECT * FROM `prod$events`.clicks
|
|
```
|
|
</CodeGroup>
|
|
|
|
In SQL the path is written as a `$`-joined schema instead; see
|
|
[Namespace](/basics/catalog#schema) for why it isn't dotted.
|
|
|
|
## Delete catalog objects {#dropping}
|
|
|
|
<Warning>
|
|
`drop_table` removes the table and every version of it. Table
|
|
[versioning](/management/table#versioning) does not survive the table being dropped, and
|
|
neither `drop_database` nor `drop_all_tables` asks for confirmation.
|
|
</Warning>
|
|
|
|
Dropping a **view** removes only the saved query. Dropping a **materialized
|
|
view** removes stored rows, because a materialized view is a table.
|