mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-12 23:02:28 +00:00
docs: keep the change-the-URI promise to the core table code
The SDK, architecture and database pages said application code moves to Enterprise by changing the URI, and the SQL page said every SDK operation has a SQL form and that the SQL reference defines the SDKs. The pages now promise what holds: start locally, then connect to Enterprise by changing the URI, and keep the same core table and search code. A new section on the SDK page lists what else changes: connection settings and credentials, missing or different methods, maintenance the deployment runs on its own configuration, and work submitted as jobs. SQL Reference is described as the dialect's specification, with links to each SDK's reference.
This commit is contained in:
@@ -114,9 +114,10 @@ terabytes.
|
||||
| Scale bound | your process | the cluster |
|
||||
| What you operate | nothing | a deployment |
|
||||
|
||||
Both expose the same table API — create, open, mutate, query — so application
|
||||
code moves between them by changing the URI. What does not move is code that
|
||||
depends on a row in the right-hand column.
|
||||
Both expose the same core table API — create, open, mutate, query — so that
|
||||
code moves between them by changing the URI. Code that depends on a row in the
|
||||
right-hand column does not; [Moving to LanceDB Enterprise](/basics/sdk#moving-to-enterprise)
|
||||
lists what changes.
|
||||
|
||||
## Choosing, and changing your mind {#choosing}
|
||||
|
||||
|
||||
@@ -269,10 +269,11 @@ where appropriate.
|
||||
|
||||
Choose [LanceDB Enterprise](/enterprise) when the resulting table becomes
|
||||
shared production infrastructure, and the workload needs distributed capacity,
|
||||
private deployment, or platform-managed operations. The underlying data format and table
|
||||
API stay the same, so the pipeline does not need to be redesigned. The agent
|
||||
instead connects to a remote `db://` table and lets the cluster handle
|
||||
maintenance and background work.
|
||||
private deployment, or platform-managed operations. The data format and the core
|
||||
table API stay the same, so the pipeline's table and search code carries over:
|
||||
the agent connects to a remote `db://` table, leaves maintenance to the
|
||||
deployment instead of calling `optimize()`, and submits long-running work as
|
||||
jobs. See [Moving to LanceDB Enterprise](/basics/sdk#moving-to-enterprise).
|
||||
|
||||
## What agents are good at here {#workflows}
|
||||
|
||||
|
||||
+46
-4
@@ -14,10 +14,13 @@ availability:
|
||||
---
|
||||
|
||||
A language SDK is the primary way to work with LanceDB: create and manage
|
||||
tables, write and query data, build indexes, run feature engineering. The same
|
||||
table API works whether you're connected to an embedded database or an
|
||||
Enterprise deployment -- application code moves between the two by changing
|
||||
the connection URI.
|
||||
tables, write and query data, build indexes, run feature engineering.
|
||||
|
||||
Start locally, then connect to Enterprise by changing your connection URI. Keep
|
||||
the same core table and search code as your application grows: creating a
|
||||
table, adding, updating and deleting rows, and searching and filtering it are
|
||||
the same calls on either connection. What else changes when you move is in
|
||||
[Moving to LanceDB Enterprise](#moving-to-enterprise).
|
||||
|
||||
## Officially supported SDKs {#supported-sdks}
|
||||
|
||||
@@ -83,6 +86,45 @@ use `spawn` instead. See
|
||||
for why.
|
||||
</Warning>
|
||||
|
||||
## Moving to LanceDB Enterprise {#moving-to-enterprise}
|
||||
|
||||
The [quickstart](/quickstart)'s core steps -- create a table, add rows, search,
|
||||
filter -- run unchanged once the connection points at a deployment. Four things
|
||||
around them change:
|
||||
|
||||
**Connection settings and credentials.** A deployment takes a `db://` URI, its
|
||||
endpoint and an API key, which its administrator gives you; keep the key in the
|
||||
environment or a secret manager. See
|
||||
[Connect to LanceDB Enterprise](/quickstart#connect-to-lancedb-enterprise), and
|
||||
[Authentication](/enterprise/authentication) for OAuth.
|
||||
|
||||
**Methods that are missing or behave differently.** A `db://` connection
|
||||
returns a remote table, which lacks some local methods and changes others. For
|
||||
example, in Python `table_exists` raises, a whole-table `to_arrow()` or
|
||||
`to_pandas()` raises -- query the table instead, as in
|
||||
`table.search().to_arrow()` -- and listing tables returns one page at a time.
|
||||
Some features also have to be enabled on the deployment before they work:
|
||||
computed columns, views, materialized views and SQL among them. Each page's
|
||||
label says where its feature applies, and
|
||||
[OSS and Enterprise](/basics/offerings) compares them all.
|
||||
|
||||
**Maintenance.** Embedded, compaction, cleanup and index updates happen when
|
||||
you call `optimize()`. On a deployment they are the deployment's work, and its
|
||||
configuration decides when they run; calling `optimize()` against it does not
|
||||
start them. Synchronous Python warns and returns `None`, while asynchronous
|
||||
Python and TypeScript raise. See [Background jobs](/maintenance/background-jobs).
|
||||
|
||||
**Long-running work.** On a deployment, building an index, refreshing a
|
||||
computed column and similar operations run on the server as
|
||||
[jobs](/basics/jobs). The `_async` form submits one and returns the job, which
|
||||
you can wait on, check later from another process, or cancel; the blocking form
|
||||
of a refresh raises there. Running on the server does not make derived data
|
||||
keep itself current: a computed column is filled only when you submit a
|
||||
refresh.
|
||||
|
||||
The [Enterprise overview](/enterprise) describes how a deployment is put
|
||||
together.
|
||||
|
||||
## What stays the same {#invariants}
|
||||
|
||||
Whichever you pick:
|
||||
|
||||
+13
-11
@@ -1,7 +1,7 @@
|
||||
---
|
||||
title: "SQL"
|
||||
sidebarTitle: "SQL"
|
||||
description: "The same tables, reached with SQL — the universal access path, and where every operation is formally documented."
|
||||
description: "The same tables, reached with SQL from any language with a Flight SQL client, and where the SQL dialect is specified."
|
||||
icon: table-list
|
||||
keywords: ["sql", "flight sql", "datafusion", "reference", "dialect"]
|
||||
availability:
|
||||
@@ -31,12 +31,12 @@ waiting on a native binding. See [SDKs](/basics/sdk) for the officially
|
||||
supported and community clients, and reach for Flight SQL whenever yours
|
||||
isn't among them.
|
||||
|
||||
## Every SDK operation, in SQL {#sdk-parity}
|
||||
## SDK operations in SQL {#sdk-parity}
|
||||
|
||||
Broadly, a public SDK method has a corresponding SQL statement -- creating a
|
||||
Many SDK operations have a SQL statement that does the same thing -- creating a
|
||||
table is `CREATE TABLE`, inserting rows is `INSERT INTO`, building an index is
|
||||
`CREATE INDEX`. The two describe the same operations, so whichever one your
|
||||
code speaks, the other is available too.
|
||||
`CREATE INDEX`. They are not interchangeable, though: each side has operations
|
||||
the other lacks.
|
||||
|
||||
| | SDK | SQL |
|
||||
| --- | :---: | :---: |
|
||||
@@ -76,8 +76,8 @@ create one with `CREATE VIEW` or by passing the query to an SDK's
|
||||
A connection to a directory or a bucket has no server to plan and execute a
|
||||
statement, and asking it to raises `NotImplementedError`. This is the largest
|
||||
behavioral difference between the embedded and deployed forms, and the reason
|
||||
the SDKs are the portable choice: SDK code moves between the two by changing
|
||||
the URI, SQL does not.
|
||||
the SDKs are the portable choice: their core table and search code moves
|
||||
between the two by changing the URI, while SQL needs the deployment.
|
||||
|
||||
```python
|
||||
db = lancedb.connect("./.lancedb") # no SQL
|
||||
@@ -117,10 +117,12 @@ SQL for the analysis on top.
|
||||
|
||||
## The complete reference {#reference}
|
||||
|
||||
This page is the concept; [SQL Reference](/sql) is the specification. It has
|
||||
one page per statement, syntax first -- and since every SDK operation has a
|
||||
SQL form, those pages end up defining the precise behavior of the SDK
|
||||
operations too, not just the SQL dialect.
|
||||
This page is the concept; [SQL Reference](/sql) is the specification of the
|
||||
SQL dialect, with one page per statement, syntax first. It does not define the
|
||||
SDKs: for a language's methods, their arguments and what they return, see that
|
||||
SDK's reference -- [Python](https://lancedb.github.io/lancedb/python/python/),
|
||||
[TypeScript](https://lancedb.github.io/lancedb/js/) or
|
||||
[Rust](https://docs.rs/lancedb/latest/lancedb/index.html).
|
||||
|
||||
<Card title="SQL Reference" icon="table-list" href="/sql">
|
||||
DDL by object, then DML, then DQL -- the detailed technical documentation
|
||||
|
||||
@@ -54,9 +54,10 @@ storage and does the work itself. The third talks to a **deployment**.
|
||||
|
||||

|
||||
|
||||
Both handles expose the same table API -- create, open, mutate, query -- so
|
||||
application code moves between them by changing the URI. What differs is where
|
||||
resolution and execution happen, and therefore which operations exist at all.
|
||||
Both handles expose the same core table API -- create, open, mutate, query --
|
||||
so that code moves between them by changing the URI. What differs is where
|
||||
resolution and execution happen, and therefore which operations exist at all;
|
||||
see [Moving to LanceDB Enterprise](/basics/sdk#moving-to-enterprise).
|
||||
|
||||
## Connecting to a deployment {#deployment}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user