diff --git a/docs/web/architecture.mdx b/docs/web/architecture.mdx index 0ef2210c3..97607991f 100644 --- a/docs/web/architecture.mdx +++ b/docs/web/architecture.mdx @@ -114,9 +114,10 @@ terabytes. | Scale bound | your process | the cluster | | What you operate | nothing | a deployment | -Both expose the same table API — create, open, mutate, query — so application -code moves between them by changing the URI. What does not move is code that -depends on a row in the right-hand column. +Both expose the same core table API — create, open, mutate, query — so that +code moves between them by changing the URI. Code that depends on a row in the +right-hand column does not; [Moving to LanceDB Enterprise](/basics/sdk#moving-to-enterprise) +lists what changes. ## Choosing, and changing your mind {#choosing} diff --git a/docs/web/basics/agentic-skills.mdx b/docs/web/basics/agentic-skills.mdx index 319e770b4..bda6e1f65 100644 --- a/docs/web/basics/agentic-skills.mdx +++ b/docs/web/basics/agentic-skills.mdx @@ -269,10 +269,11 @@ where appropriate. Choose [LanceDB Enterprise](/enterprise) when the resulting table becomes shared production infrastructure, and the workload needs distributed capacity, -private deployment, or platform-managed operations. The underlying data format and table -API stay the same, so the pipeline does not need to be redesigned. The agent -instead connects to a remote `db://` table and lets the cluster handle -maintenance and background work. +private deployment, or platform-managed operations. The data format and the core +table API stay the same, so the pipeline's table and search code carries over: +the agent connects to a remote `db://` table, leaves maintenance to the +deployment instead of calling `optimize()`, and submits long-running work as +jobs. See [Moving to LanceDB Enterprise](/basics/sdk#moving-to-enterprise). ## What agents are good at here {#workflows} diff --git a/docs/web/basics/sdk.mdx b/docs/web/basics/sdk.mdx index 8568107ac..c25bbcaa7 100644 --- a/docs/web/basics/sdk.mdx +++ b/docs/web/basics/sdk.mdx @@ -14,10 +14,13 @@ availability: --- A language SDK is the primary way to work with LanceDB: create and manage -tables, write and query data, build indexes, run feature engineering. The same -table API works whether you're connected to an embedded database or an -Enterprise deployment -- application code moves between the two by changing -the connection URI. +tables, write and query data, build indexes, run feature engineering. + +Start locally, then connect to Enterprise by changing your connection URI. Keep +the same core table and search code as your application grows: creating a +table, adding, updating and deleting rows, and searching and filtering it are +the same calls on either connection. What else changes when you move is in +[Moving to LanceDB Enterprise](#moving-to-enterprise). ## Officially supported SDKs {#supported-sdks} @@ -83,6 +86,45 @@ use `spawn` instead. See for why. +## Moving to LanceDB Enterprise {#moving-to-enterprise} + +The [quickstart](/quickstart)'s core steps -- create a table, add rows, search, +filter -- run unchanged once the connection points at a deployment. Four things +around them change: + +**Connection settings and credentials.** A deployment takes a `db://` URI, its +endpoint and an API key, which its administrator gives you; keep the key in the +environment or a secret manager. See +[Connect to LanceDB Enterprise](/quickstart#connect-to-lancedb-enterprise), and +[Authentication](/enterprise/authentication) for OAuth. + +**Methods that are missing or behave differently.** A `db://` connection +returns a remote table, which lacks some local methods and changes others. For +example, in Python `table_exists` raises, a whole-table `to_arrow()` or +`to_pandas()` raises -- query the table instead, as in +`table.search().to_arrow()` -- and listing tables returns one page at a time. +Some features also have to be enabled on the deployment before they work: +computed columns, views, materialized views and SQL among them. Each page's +label says where its feature applies, and +[OSS and Enterprise](/basics/offerings) compares them all. + +**Maintenance.** Embedded, compaction, cleanup and index updates happen when +you call `optimize()`. On a deployment they are the deployment's work, and its +configuration decides when they run; calling `optimize()` against it does not +start them. Synchronous Python warns and returns `None`, while asynchronous +Python and TypeScript raise. See [Background jobs](/maintenance/background-jobs). + +**Long-running work.** On a deployment, building an index, refreshing a +computed column and similar operations run on the server as +[jobs](/basics/jobs). The `_async` form submits one and returns the job, which +you can wait on, check later from another process, or cancel; the blocking form +of a refresh raises there. Running on the server does not make derived data +keep itself current: a computed column is filled only when you submit a +refresh. + +The [Enterprise overview](/enterprise) describes how a deployment is put +together. + ## What stays the same {#invariants} Whichever you pick: diff --git a/docs/web/basics/sql.mdx b/docs/web/basics/sql.mdx index 1a348e304..f7f6a39a2 100644 --- a/docs/web/basics/sql.mdx +++ b/docs/web/basics/sql.mdx @@ -1,7 +1,7 @@ --- title: "SQL" sidebarTitle: "SQL" -description: "The same tables, reached with SQL — the universal access path, and where every operation is formally documented." +description: "The same tables, reached with SQL from any language with a Flight SQL client, and where the SQL dialect is specified." icon: table-list keywords: ["sql", "flight sql", "datafusion", "reference", "dialect"] availability: @@ -31,12 +31,12 @@ waiting on a native binding. See [SDKs](/basics/sdk) for the officially supported and community clients, and reach for Flight SQL whenever yours isn't among them. -## Every SDK operation, in SQL {#sdk-parity} +## SDK operations in SQL {#sdk-parity} -Broadly, a public SDK method has a corresponding SQL statement -- creating a +Many SDK operations have a SQL statement that does the same thing -- creating a table is `CREATE TABLE`, inserting rows is `INSERT INTO`, building an index is -`CREATE INDEX`. The two describe the same operations, so whichever one your -code speaks, the other is available too. +`CREATE INDEX`. They are not interchangeable, though: each side has operations +the other lacks. | | SDK | SQL | | --- | :---: | :---: | @@ -76,8 +76,8 @@ create one with `CREATE VIEW` or by passing the query to an SDK's A connection to a directory or a bucket has no server to plan and execute a statement, and asking it to raises `NotImplementedError`. This is the largest behavioral difference between the embedded and deployed forms, and the reason -the SDKs are the portable choice: SDK code moves between the two by changing -the URI, SQL does not. +the SDKs are the portable choice: their core table and search code moves +between the two by changing the URI, while SQL needs the deployment. ```python db = lancedb.connect("./.lancedb") # no SQL @@ -117,10 +117,12 @@ SQL for the analysis on top. ## The complete reference {#reference} -This page is the concept; [SQL Reference](/sql) is the specification. It has -one page per statement, syntax first -- and since every SDK operation has a -SQL form, those pages end up defining the precise behavior of the SDK -operations too, not just the SQL dialect. +This page is the concept; [SQL Reference](/sql) is the specification of the +SQL dialect, with one page per statement, syntax first. It does not define the +SDKs: for a language's methods, their arguments and what they return, see that +SDK's reference -- [Python](https://lancedb.github.io/lancedb/python/python/), +[TypeScript](https://lancedb.github.io/lancedb/js/) or +[Rust](https://docs.rs/lancedb/latest/lancedb/index.html). DDL by object, then DML, then DQL -- the detailed technical documentation diff --git a/docs/web/management/database.mdx b/docs/web/management/database.mdx index c29b473cd..680452c20 100644 --- a/docs/web/management/database.mdx +++ b/docs/web/management/database.mdx @@ -54,9 +54,10 @@ storage and does the work itself. The third talks to a **deployment**. ![](/static/assets/images/overview/understanding-tables.png) -Both handles expose the same table API -- create, open, mutate, query -- so -application code moves between them by changing the URI. What differs is where -resolution and execution happen, and therefore which operations exist at all. +Both handles expose the same core table API -- create, open, mutate, query -- +so that code moves between them by changing the URI. What differs is where +resolution and execution happen, and therefore which operations exist at all; +see [Moving to LanceDB Enterprise](/basics/sdk#moving-to-enterprise). ## Connecting to a deployment {#deployment}