mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 22:26:53 +00:00
A broad documentation pass across the areas LanceDB Enterprise and Embedded diverge on most: **Data Curation** -- GPU Function execution, Slurm scheduling (and what Slurm is, for readers new to it), blob late materialization as its own page, joins, aggregation/profiling, sampling, near-duplicate detection, chunking, distributed shuffle, and IVF-partition clustering. **Feature Engineering** -- computed columns and materialized views, restructured to lead with the Function-based path and show the SQL form side by side with Python throughout; a single Auto Refresh page instead of duplicating scheduling details on every refresh page; adaptive execution; checkpointing; GPU and Slurm placement for refresh workers; a Training Cache page for the PyTorch data loader's automatic cache benefit. **Security & Governance** -- an Audit Log reference (schema, access, grants), Secrets promoted to an ordinary catalog object with their own SQL DDL and a vending/rotation guide, and privilege-table fixes verified against the authorization source. **Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry moved into a real Deployment area; Automated Cleanup folded into Intelligent Auto-Maintenance (every other maintenance decision already lived there); RTE Compaction and Advanced Conflict Resolution documented in depth. **Landing page** -- the Enterprise feature comparison table reorganized (several redundant or stale rows removed, several new areas added), and the "why Enterprise" bullets replaced with ones tied to a concrete page (Performance, Scale, Fault tolerance, Multicloud deployment, Simplify operations) instead of vague promises. Plus a full pass addressing lancedb-gatekeeper's review: a real test-safety bug (a quickstart test could silently overwrite and delete a deployment's own `frames` table), a fabricated SQL statement with no backing implementation, an actual dedup-guarantee bug in `take_row_ids`, and dozens of documentation-accuracy fixes verified against source rather than taken on faith -- each either fixed, or left with an explicit note on why it needs a feature or deployment owner's sign-off instead of being guessed at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
175 lines
5.9 KiB
Plaintext
175 lines
5.9 KiB
Plaintext
---
|
|
title: "Secret"
|
|
sidebarTitle: "Secret"
|
|
description: "CREATE, ALTER, DROP, SHOW and DESCRIBE SECRET, and granting privileges on Secrets."
|
|
icon: key
|
|
keywords: ["secret", "create secret", "alter secret", "drop secret", "show secrets", "describe secret", "grant", "sql"]
|
|
---
|
|
|
|
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
|
|
|
|
A [Secret](/management/secret) is a named credential that Functions bind by
|
|
reference. These statements store, rotate, list, describe and drop Secrets.
|
|
|
|
A Secret is addressed like a table: `[db.][namespace.]name`. An unqualified
|
|
name uses the session's database and namespace. A nested namespace is one
|
|
backtick-quoted name with `$` between its components, and `public` is the
|
|
database's root namespace. A Secret name may contain letters, digits, `_`, `.`
|
|
and `-`, so quote one that contains `.` or `-`:
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
api_key
|
|
platform.`openai-prod`
|
|
analytics.`ml$vision`.`openai-prod`
|
|
```
|
|
</CodeGroup>
|
|
|
|
`CREATE`, `ALTER` and `DROP` return one row of `operation`, `name` and
|
|
`status`. Without `IF [NOT] EXISTS`, a taken name (for `CREATE`) or a missing
|
|
Secret (for `ALTER` and `DROP`) is an error rather than a status.
|
|
|
|
## CREATE SECRET {#create}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
CREATE SECRET platform.`openai-prod` TYPE = GENERIC_STRING SECRET_STRING = 'sk-...'
|
|
CREATE SECRET IF NOT EXISTS api_key TYPE = GENERIC_STRING SECRET_STRING = $$sk-...$$
|
|
```
|
|
</CodeGroup>
|
|
|
|
Stores a new Secret. `TYPE = GENERIC_STRING` is required; it is currently the
|
|
only supported type. The value may be single-quoted or dollar-quoted, or a
|
|
bound `$1` parameter -- the same choice `ALTER SECRET` offers below -- so the
|
|
value never has to appear in the SQL text itself:
|
|
|
|
<CodeGroup>
|
|
```python Python icon="python"
|
|
db.execute_query(
|
|
"CREATE SECRET platform.`openai-prod` TYPE = GENERIC_STRING SECRET_STRING = $1",
|
|
parameters=["sk-..."],
|
|
)
|
|
```
|
|
</CodeGroup>
|
|
|
|
Requires `CREATE SECRET` on the namespace; the caller becomes the Secret's
|
|
owner.
|
|
|
|
The status is `created`, or `already_exists` with `IF NOT EXISTS`. It never
|
|
overwrites an existing Secret; use `ALTER SECRET`.
|
|
|
|
## ALTER SECRET {#alter}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
ALTER SECRET platform.`openai-prod` SET SECRET_STRING = 'sk-...'
|
|
ALTER SECRET IF EXISTS api_key SET SECRET_STRING = $$sk-...$$
|
|
```
|
|
</CodeGroup>
|
|
|
|
Replaces the Secret's value -- `SECRET_STRING` takes the same single-quoted,
|
|
dollar-quoted, or bound `$1` parameter forms `CREATE SECRET` does. Every
|
|
Function bound to it gets the latest value the next time a job starts -- a
|
|
job already running, and its retries, keep the value they captured at their
|
|
own start -- without a new Function version. Requires `UPDATE` on the
|
|
Secret.
|
|
|
|
The status is `altered`, or `not_found` with `IF EXISTS`.
|
|
|
|
## DROP SECRET {#drop}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
DROP SECRET platform.`openai-prod`
|
|
DROP SECRET IF EXISTS api_key
|
|
```
|
|
</CodeGroup>
|
|
|
|
Deletes the Secret and every grant on it. Functions bound to it fail at their
|
|
next job, naming the Secret. Requires `OWNERSHIP` on the Secret.
|
|
|
|
The status is `dropped`, or `not_found` with `IF EXISTS`.
|
|
|
|
<Warning>
|
|
`DROP SECRET` cannot be undone. A new Secret created under the same name is
|
|
picked up by every Function still bound to that name.
|
|
</Warning>
|
|
|
|
## SHOW SECRETS {#show}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
SHOW SECRETS
|
|
SHOW SECRETS IN platform
|
|
SHOW SECRETS IN analytics.`ml$vision`
|
|
SHOW SECRETS IN public
|
|
SHOW SECRETS IN platform WHERE name > 'openai-prod' LIMIT 50
|
|
```
|
|
</CodeGroup>
|
|
|
|
Lists the Secret names in one namespace as a sorted `name` column. Without
|
|
`IN`, it lists the session's namespace; `IN public` lists the root. `WHERE
|
|
name > '<cursor>'` pages past the last name already seen, and `LIMIT`
|
|
bounds the page size; `WHERE` must compare `name` with `>`. Requires
|
|
`USAGE` on the database and the namespace.
|
|
|
|
## DESCRIBE SECRET {#describe}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
DESCRIBE SECRET platform.`openai-prod`
|
|
DESC SECRET api_key
|
|
```
|
|
</CodeGroup>
|
|
|
|
Returns one row:
|
|
|
|
| Column | Type | Meaning |
|
|
| --- | --- | --- |
|
|
| `name` | string | The Secret's name |
|
|
| `created_at_millis` | int64 | When it was created, in milliseconds since the Unix epoch |
|
|
| `updated_at_millis` | int64 | When its value was last set. Watch this change to confirm a rotation landed. |
|
|
| `history` | list of struct `{generation: uint64, at_millis: int64}` | Creation and each rotation since, oldest first, up to the most recent 100 |
|
|
|
|
A missing Secret is an error. Requires `USAGE` on the Secret.
|
|
|
|
## GRANT and REVOKE {#grant}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
GRANT CREATE SECRET ON SCHEMA platform TO ROLE platform_admins
|
|
GRANT USAGE, UPDATE ON SECRET platform.`openai-prod` TO ROLE etl, rotation_service
|
|
REVOKE UPDATE ON SECRET platform.`openai-prod` FROM ROLE etl
|
|
```
|
|
</CodeGroup>
|
|
|
|
`CREATE SECRET` on the containing `SCHEMA` creates one; `OWNERSHIP`, `USAGE`
|
|
(bind it to a Function) and `UPDATE` (rotate it) apply to the Secret itself.
|
|
See [GRANT and REVOKE](/sql/ddl/grant) for the full statement grammar, every
|
|
target type, and the shared mechanics -- grantee forms, no-op rules, and the
|
|
status a statement returns -- which apply the same way to a Secret as to
|
|
any other object.
|
|
|
|
## The value stays out of output {#redaction}
|
|
|
|
A Secret's value never appears in a statement's result, an error message,
|
|
`EXPLAIN` output, or the query log. The query log masks everything after the
|
|
name in `CREATE SECRET`, and everything after `SET` in `ALTER SECRET`.
|
|
|
|
<Note>
|
|
SQL has no syntax for binding a Secret to a Function. Register Functions that
|
|
bind Secrets through the Python SDK; see
|
|
[Vend a Secret to a Function](/curation/vend-secret).
|
|
</Note>
|
|
|
|
## What to read next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Secret" icon="key" href="/management/secret">
|
|
Create, rotate, list and drop the credential from the Python SDK and SQL.
|
|
</Card>
|
|
<Card title="Privileges" icon="key" href="/governance/privileges#sql">
|
|
Every object's privileges, and how grants resolve.
|
|
</Card>
|
|
</CardGroup>
|