mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 14:16:54 +00:00
A broad documentation pass across the areas LanceDB Enterprise and Embedded diverge on most: **Data Curation** -- GPU Function execution, Slurm scheduling (and what Slurm is, for readers new to it), blob late materialization as its own page, joins, aggregation/profiling, sampling, near-duplicate detection, chunking, distributed shuffle, and IVF-partition clustering. **Feature Engineering** -- computed columns and materialized views, restructured to lead with the Function-based path and show the SQL form side by side with Python throughout; a single Auto Refresh page instead of duplicating scheduling details on every refresh page; adaptive execution; checkpointing; GPU and Slurm placement for refresh workers; a Training Cache page for the PyTorch data loader's automatic cache benefit. **Security & Governance** -- an Audit Log reference (schema, access, grants), Secrets promoted to an ordinary catalog object with their own SQL DDL and a vending/rotation guide, and privilege-table fixes verified against the authorization source. **Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry moved into a real Deployment area; Automated Cleanup folded into Intelligent Auto-Maintenance (every other maintenance decision already lived there); RTE Compaction and Advanced Conflict Resolution documented in depth. **Landing page** -- the Enterprise feature comparison table reorganized (several redundant or stale rows removed, several new areas added), and the "why Enterprise" bullets replaced with ones tied to a concrete page (Performance, Scale, Fault tolerance, Multicloud deployment, Simplify operations) instead of vague promises. Plus a full pass addressing lancedb-gatekeeper's review: a real test-safety bug (a quickstart test could silently overwrite and delete a deployment's own `frames` table), a fabricated SQL statement with no backing implementation, an actual dedup-guarantee bug in `take_row_ids`, and dozens of documentation-accuracy fixes verified against source rather than taken on faith -- each either fixed, or left with an explicit note on why it needs a feature or deployment owner's sign-off instead of being guessed at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
299 lines
8.5 KiB
Plaintext
299 lines
8.5 KiB
Plaintext
---
|
|
title: "LanceDB CLI"
|
|
sidebarTitle: "CLI"
|
|
description: "Connect to LanceDB, run SQL, and manage connection profiles with the lancedb command-line client."
|
|
icon: terminal
|
|
keywords: ["cli", "command line", "sql", "profiles", "authentication"]
|
|
---
|
|
|
|
Use the `lancedb` command-line client to open an interactive SQL shell, run a
|
|
statement, or execute a SQL file. SQL requires a running deployment with an
|
|
[Arrow Flight SQL endpoint](/basics/sql#sql-is-remote).
|
|
|
|
## Install
|
|
|
|
Download a CLI archive and `SHA256SUMS` from the
|
|
[LanceDB releases page](https://github.com/lancedb/lancedb/releases). The CLI
|
|
shares the SDK release version, including beta versions.
|
|
|
|
| Platform | Target | Requirements |
|
|
| --- | --- | --- |
|
|
| Linux x86_64 | `x86_64-unknown-linux-gnu` | glibc 2.28 or newer; Haswell-compatible CPU with AVX2, FMA, and F16C |
|
|
| Linux ARM64 | `aarch64-unknown-linux-gnu` | glibc 2.28 or newer |
|
|
| macOS Apple Silicon | `aarch64-apple-darwin` | macOS 14 or newer |
|
|
|
|
Set the version and target to match your chosen release and platform:
|
|
|
|
```sh
|
|
CLI_VERSION=X.Y.Z # Replace with the release version, for example X.Y.Z-beta.N
|
|
CLI_TARGET=aarch64-apple-darwin
|
|
CLI_STEM="lancedb-v${CLI_VERSION}-${CLI_TARGET}"
|
|
CLI_RELEASE_URL="https://github.com/lancedb/lancedb/releases/download/v${CLI_VERSION}"
|
|
curl -fLO "${CLI_RELEASE_URL}/${CLI_STEM}.tar.gz"
|
|
curl -fLO "${CLI_RELEASE_URL}/SHA256SUMS"
|
|
awk -v archive="${CLI_STEM}.tar.gz" '$2 == archive' SHA256SUMS > "${CLI_STEM}.sha256"
|
|
```
|
|
|
|
Verify the downloaded archive before extracting it. On Linux:
|
|
|
|
```sh
|
|
sha256sum --check "${CLI_STEM}.sha256"
|
|
```
|
|
|
|
On macOS:
|
|
|
|
```sh
|
|
shasum -a 256 --check "${CLI_STEM}.sha256"
|
|
```
|
|
|
|
After the checksum passes, install the executable into a directory on your `PATH`:
|
|
|
|
```sh
|
|
tar -xzf "${CLI_STEM}.tar.gz"
|
|
mkdir -p "$HOME/.local/bin"
|
|
install -m 755 "${CLI_STEM}/lancedb" "$HOME/.local/bin/lancedb"
|
|
export PATH="$HOME/.local/bin:$PATH"
|
|
lancedb --version
|
|
```
|
|
|
|
Add `$HOME/.local/bin` to your shell's `PATH` configuration to use it in future
|
|
sessions. Each archive also contains the license and Rust third-party notices.
|
|
|
|
### Build from source
|
|
|
|
With Rust and Git installed, build the CLI from the LanceDB repository. Check
|
|
out a release tag to build that version:
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
git clone https://github.com/lancedb/lancedb.git
|
|
cd lancedb
|
|
git checkout vX.Y.Z # Replace with the desired release tag
|
|
cargo install --locked --path rust/lancedb/cli
|
|
```
|
|
</CodeGroup>
|
|
|
|
Check the available options:
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb --help
|
|
```
|
|
</CodeGroup>
|
|
|
|
## Connect to a deployment
|
|
|
|
Get the Flight SQL endpoint, HTTP API endpoint, database name, and credentials
|
|
from your deployment administrator. Create a named profile to save the
|
|
connection settings:
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb profile create production \
|
|
--sql-server-url grpcs://sql.example.com:10025 \
|
|
--http-server-url https://query.example.com \
|
|
--database analytics
|
|
```
|
|
</CodeGroup>
|
|
|
|
The SQL endpoint executes statements. The HTTP endpoint supports shell commands
|
|
that inspect table and index metadata. Replace both example URLs with the
|
|
endpoints for your deployment; they may use different hosts or ports.
|
|
|
|
For API-key authentication, set the credential in your environment:
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
export LANCEDB_API_KEY="YOUR_API_KEY"
|
|
lancedb --profile production
|
|
```
|
|
</CodeGroup>
|
|
|
|
Use `grpcs://` or `https://` for a TLS-protected SQL connection. A bare
|
|
`host:port` or a `grpc://` address selects a plaintext connection.
|
|
|
|
### Set the default profile
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb profile set production
|
|
lancedb
|
|
```
|
|
</CodeGroup>
|
|
|
|
<Note>
|
|
Running `lancedb` opens the SQL shell when the default profile has a Flight SQL
|
|
endpoint. If no endpoint is configured, it prints usage and connection setup
|
|
guidance. Command and file execution also require a configured endpoint.
|
|
</Note>
|
|
|
|
To connect to an existing server on localhost, explicitly select the built-in
|
|
local profile:
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb --profile local
|
|
```
|
|
</CodeGroup>
|
|
|
|
This profile uses Flight SQL on port `10025` and HTTP on port `10024`.
|
|
|
|
## Run SQL
|
|
|
|
Execute a statement and exit:
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb --profile production --command 'SELECT 1' --format json
|
|
```
|
|
</CodeGroup>
|
|
|
|
To execute a file, separate its statements with semicolons. For example, save
|
|
the following as `queries.sql`:
|
|
|
|
<CodeGroup>
|
|
```sql queries.sql icon="database"
|
|
SELECT COUNT(*) FROM documents;
|
|
SELECT id, text FROM documents LIMIT 10;
|
|
```
|
|
</CodeGroup>
|
|
|
|
Run the file with:
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb --profile production --file queries.sql
|
|
```
|
|
</CodeGroup>
|
|
|
|
Quoted strings and comments in SQL files are preserved when statements are
|
|
split. `--command` and `--file` are mutually exclusive.
|
|
|
|
### Choose an output format
|
|
|
|
For query results, `--format` accepts the following values:
|
|
|
|
| Format | Output |
|
|
| --- | --- |
|
|
| `table` | A display table, the default |
|
|
| `csv` | Comma-separated values with column headers |
|
|
| `json` | A JSON array of rows |
|
|
| `ndjson` | One JSON row per line |
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb --profile production --command 'SELECT id FROM documents' --format csv
|
|
```
|
|
</CodeGroup>
|
|
|
|
## Use the SQL shell
|
|
|
|
End SQL statements with a semicolon. Enter `\?` to see the shell's commands.
|
|
|
|
| Command | Purpose |
|
|
| --- | --- |
|
|
| `\dt` | List tables |
|
|
| `\d documents` | Show a table's columns and indexes |
|
|
| `\di` | List indexes |
|
|
| `\dj` | List jobs |
|
|
| `\c analytics` | Connect to another database |
|
|
| `\e` | Edit the query buffer in an external editor |
|
|
| `\watch 2` | Repeat the last query or listing every two seconds |
|
|
| `\q` | Exit the shell |
|
|
|
|
Add `+` to `\dt`, `\d`, or `\di` for more detail. Use Ctrl+C to cancel the
|
|
current input or stop watching a query, and Ctrl+D to exit.
|
|
|
|
Large interactive results use a pager. Set `NO_PAGER=1` to disable paging.
|
|
|
|
### Inspect jobs with SQL
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb --profile production --command 'SHOW JOBS'
|
|
lancedb --profile production --command "SHOW ERRORS ON documents FOR JOB 'JOB_ID'"
|
|
lancedb --profile production --command "CANCEL JOB 'JOB_ID'"
|
|
```
|
|
</CodeGroup>
|
|
|
|
See the [job SQL reference](/sql/ddl/job) for syntax and deployment requirements,
|
|
and the [SQL reference](/sql) for other supported statements.
|
|
|
|
## Manage profiles
|
|
|
|
Profiles are stored in `~/.lancedb/config.toml`. The CLI reuses existing
|
|
Enterprise profiles and cached OAuth sessions.
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb profile list
|
|
lancedb profile show production
|
|
lancedb profile edit production --database staging
|
|
lancedb profile delete production
|
|
```
|
|
</CodeGroup>
|
|
|
|
Use `lancedb profile create NAME` without connection flags for interactive
|
|
setup. For a single SQL invocation, flags such as `--database` and
|
|
`--sql-server-url` override the selected profile.
|
|
|
|
Put each command's flags after that command. For example, use
|
|
`lancedb auth login --profile production` or `lancedb profile edit production
|
|
--database staging`.
|
|
|
|
## Authenticate with OAuth
|
|
|
|
To use OAuth, configure an `oidc` table in the selected profile with your
|
|
identity provider's settings. For example, add the following to
|
|
`~/.lancedb/config.toml` for a device-code login:
|
|
|
|
```toml
|
|
[profiles.production.oidc]
|
|
flow = "device_code"
|
|
issuer_url = "https://identity.example.com"
|
|
client_id = "YOUR_CLIENT_ID"
|
|
scopes = ["openid", "offline_access"]
|
|
```
|
|
|
|
Choose one authentication method per profile. An OAuth profile cannot also
|
|
contain `api_key` or `oidc_access_token_file`. Environment credentials override
|
|
the profile, so clear API-key overrides before connecting with OAuth:
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
unset LANCEDB_API_KEY
|
|
lancedb auth login --profile production
|
|
lancedb auth status --profile production
|
|
lancedb --profile production
|
|
```
|
|
</CodeGroup>
|
|
|
|
`auth login` follows the profile's OAuth flow. Browser and device-code sessions
|
|
cache refresh tokens for later commands. Use `lancedb auth logout --profile
|
|
production` to clear the profile's cached session.
|
|
|
|
See [authentication](/enterprise/authentication) for identity provider setup and
|
|
available authentication methods.
|
|
|
|
## Configure TLS
|
|
|
|
To trust a private certificate authority, pass its PEM certificate:
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb --profile production --ca-cert /path/to/ca.pem
|
|
```
|
|
</CodeGroup>
|
|
|
|
If the server requires mutual TLS, supply both a client certificate and its
|
|
PKCS#8 private key. Save these settings in the profile with:
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
lancedb profile edit production \
|
|
--ca-cert /path/to/ca.pem \
|
|
--client-cert /path/to/client.pem \
|
|
--client-key /path/to/client-key.pem
|
|
```
|
|
</CodeGroup>
|