From ac2db48af6740d2ea7749abb9c623352361e8c74 Mon Sep 17 00:00:00 2001 From: Brendan Clement Date: Thu, 8 Oct 2026 21:31:42 -0700 Subject: [PATCH] docs: declare where SQL and the SDKs apply SQL is unavailable in the embedded library and varies on a deployment, which serves it only when its Flight SQL endpoint is enabled and reachable, and the SDK has to be told where that is. The page's opening badge goes. The SDKs page declares both offerings. Neither SDK aggregates or joins, so the parity tables no longer say they do, and views can be created through the SDKs on a deployment. Pages that said vector search has no SQL form now point to the version-dependent SQL form, and COPY, which newer deployment versions refuse, is marked as such in the reference. --- docs/web/basics/query.mdx | 6 ++++-- docs/web/basics/sdk.mdx | 18 +++++++++++++----- docs/web/basics/sql.mdx | 28 +++++++++++++++++++++------- docs/web/sql/dml/copy.mdx | 6 +++++- docs/web/sql/dql/aggregation.mdx | 9 +++++---- docs/web/sql/index.mdx | 15 ++++++++------- docs/web/sql/operators.mdx | 4 ++-- 7 files changed, 58 insertions(+), 28 deletions(-) diff --git a/docs/web/basics/query.mdx b/docs/web/basics/query.mdx index 449c9b498..1447c4489 100644 --- a/docs/web/basics/query.mdx +++ b/docs/web/basics/query.mdx @@ -83,8 +83,10 @@ return results. They are fast enough not to need a handle. See Query-builder searches -- [vector](/search/vector-search), [full-text](/search/full-text-search), [hybrid](/search/hybrid-search) -- return -results directly. They are fast enough not to need a handle, and they have no -SQL form. See [Semantic Search](/search). +results directly. They are fast enough not to need a handle. In SQL on a +deployment, full-text search is the [`fts()`](/sql/functions#fts) table +function and vector search [depends on the deployment](/basics/sql#no-vector-sql); +hybrid search has no SQL form. See [Semantic Search](/search). ## When the work outlives the call {#jobs} diff --git a/docs/web/basics/sdk.mdx b/docs/web/basics/sdk.mdx index 3f8170e11..28d094d09 100644 --- a/docs/web/basics/sdk.mdx +++ b/docs/web/basics/sdk.mdx @@ -4,6 +4,13 @@ sidebarTitle: "SDKs" description: "The language clients — Python, TypeScript, Rust — and what stays true across all of them." icon: code-compare keywords: ["python", "typescript", "rust", "java", "sdk", "rest", "api reference"] +availability: + oss: available + enterprise: available + summary: >- + The Python, TypeScript and Rust SDKs work with embedded databases and, + through `db://`, with Enterprise deployments; the Java SDK reaches + deployments only. What each can do differs by language and connection. --- A language SDK is the primary way to work with LanceDB: create and manage @@ -29,14 +36,15 @@ Not every SDK covers the same surface, though: | --- | :---: | :---: | | Create and manage catalog objects | yes | yes | | Insert, merge, update, delete | yes | yes | -| Filter, project, aggregate, join | yes | yes | +| Filter, project | yes | yes | | Vector search | yes | yes | | Full-text search | yes | yes | | Feature engineering, jobs | yes | partly | | Runs against a local database | yes | yes | -TypeScript has no methods for [functions](/management/function). Reach the -gap with SQL instead -- see [SQL](/basics/sql). +Neither SDK aggregates or joins, and TypeScript has no methods for +[functions](/management/function). Reach those with SQL on a deployment +instead -- see [SQL](/basics/sql). ## REST API {#rest} @@ -62,8 +70,8 @@ and release cadence vary. | [R](https://github.com/CathalByrneGit/lancedb) | Community-contributed R package | | [Flutter](https://github.com/Alexcn/flutter_lancedb) | Community-contributed Flutter bindings | -For any other language, [SQL](/basics/sql) reaches the same tables without a -LanceDB-specific SDK at all. +For any other language, [SQL](/basics/sql) on a deployment reaches the same +tables without a LanceDB-specific SDK at all. ## Python multiprocessing {#multiprocessing} diff --git a/docs/web/basics/sql.mdx b/docs/web/basics/sql.mdx index 6b74486eb..1a348e304 100644 --- a/docs/web/basics/sql.mdx +++ b/docs/web/basics/sql.mdx @@ -4,11 +4,14 @@ sidebarTitle: "SQL" description: "The same tables, reached with SQL — the universal access path, and where every operation is formally documented." icon: table-list keywords: ["sql", "flight sql", "datafusion", "reference", "dialect"] +availability: + oss: unavailable + enterprise: varies + summary: >- + SQL runs on an Enterprise deployment whose Flight SQL endpoint is enabled + and reachable; embedded connections raise instead. --- -Enterprise SQL runs on a deployment. A local -database has no engine to plan a statement, so `execute_query` raises there. - SQL is a second way to reach the same tables the [SDKs](/basics/sdk) do -- not a different product, a different interface onto one engine. A table written from Python is queried from SQL without an export step, and a table @@ -39,12 +42,17 @@ code speaks, the other is available too. | --- | :---: | :---: | | Create and manage catalog objects | yes | partly | | Insert, merge, update, delete | yes | yes | -| Filter, project, aggregate, join | yes | yes | +| Filter, project | yes | yes | +| Aggregate, join | no | yes | | Vector search | yes | version-dependent | | Full-text search | yes | yes | | Feature engineering, jobs | yes | yes | | Runs against a local database | yes | **no** | +On a deployment, feature engineering must also be enabled by the deployment, +whether it is declared in SQL or through an SDK -- see +[Computed columns](/feature-engineering/add-computed-column). + ## Vector search depends on the deployment {#no-vector-sql} The SDK query builder provides vector search across embedded and deployed @@ -59,9 +67,9 @@ Full-text search is available through the [`fts()`](/sql/functions#fts) table function and the [`contains_tokens`](/sql/scalar-functions#contains-tokens) predicate. -The reverse gap also exists: a [view](/management/view) is a saved SQL query, -so SQL is the only way to create one -- there's nothing for an SDK method to -call. +A [view](/management/view) is a saved SQL query. On a deployment you can +create one with `CREATE VIEW` or by passing the query to an SDK's +`create_view`; an embedded database has no views. ## SQL needs a deployment {#sql-is-remote} @@ -76,6 +84,12 @@ db = lancedb.connect("./.lancedb") # no SQL db = lancedb.connect("db://my-database") # SQL ``` +A deployment serves SQL only when its Flight SQL endpoint is enabled and +reachable, which is part of the deployment's configuration. The SDK also has to +know where that endpoint is: pass `sql_host_override` when connecting, or +connect through an `http://` `host_override`, from which the SDK derives it +(the next port up, or 10025 when the override names no port). + ## Running SQL from Python {#sql-from-python} ```python diff --git a/docs/web/sql/dml/copy.mdx b/docs/web/sql/dml/copy.mdx index 837b92285..3fe002fe8 100644 --- a/docs/web/sql/dml/copy.mdx +++ b/docs/web/sql/dml/copy.mdx @@ -6,7 +6,11 @@ icon: file-export keywords: ["copy", "export", "parquet", "sql"] --- -Enterprise [SQL runs on a deployment.](/sql) + +Newer deployment versions refuse `COPY` over the SQL endpoint with "COPY is not +supported". On those, export a result by reading it through a Flight SQL client +or an SDK and writing the files from your application. + ```sql COPY (SELECT id, title FROM documents) TO 's3://bucket/export/' STORED AS PARQUET diff --git a/docs/web/sql/dql/aggregation.mdx b/docs/web/sql/dql/aggregation.mdx index 2bd8ddd58..82237ff76 100644 --- a/docs/web/sql/dql/aggregation.mdx +++ b/docs/web/sql/dql/aggregation.mdx @@ -112,7 +112,8 @@ estimate -- a 1% sample of a skewed column will miss rare categories entirely. ## What aggregation does not do {#not} -Aggregates run over rows the query already produced. They cannot combine with -[vector search](/search/vector-search) in one statement, because similarity -search has no SQL form. Run the search in an SDK and aggregate its result, or -materialize the search output first. +Aggregates run over rows the query already produced. Whether a statement can +include [vector search](/search/vector-search) +[depends on the deployment](/basics/sql#no-vector-sql); where it cannot, run the +search in an SDK and aggregate its result, or materialize the search output +first. diff --git a/docs/web/sql/index.mdx b/docs/web/sql/index.mdx index 2f00b126a..4b92ef38c 100644 --- a/docs/web/sql/index.mdx +++ b/docs/web/sql/index.mdx @@ -6,9 +6,9 @@ icon: table-list keywords: ["sql", "datafusion", "flight sql", "reference", "dialect"] --- -Enterprise SQL runs on a deployment. A local database -has no engine to plan a statement, so `execute_query` raises there -- see -[SQL](/basics/sql#sql-is-remote). +Enterprise SQL runs on a deployment that enables its +Flight SQL endpoint. A local database has no SQL endpoint, so `execute_query` +raises there -- see [SQL](/basics/sql#sql-is-remote). This is a reference rather than a guide: one page per construct, syntax first. The guides elsewhere in these docs show the same operations from the SDKs. @@ -87,8 +87,8 @@ const result = await client.query("SELECT * FROM flights WHERE origin = 'SFO'"); The endpoint executes one statement per request and is meant for the queries and DDL/DML pages below. Reach for the SDKs instead for table-management -operations that have no SQL form, such as building a vector index or a merge -insert -- see [What SQL cannot do](#cannot). +operations that have no SQL form, such as a merge insert -- see +[What SQL cannot do](#cannot). ## Reference {#reference} @@ -111,7 +111,7 @@ insert -- see [What SQL cannot do](#cannot). | [Insert](/sql/dml/insert) | `INSERT INTO`, including `INSERT ... SELECT` | | [Update](/sql/dml/update) | `UPDATE ... SET` | | [Delete](/sql/dml/delete) | `DELETE FROM` | -| [Copy](/sql/dml/copy) | `COPY INTO` / `COPY TO` | +| [Copy](/sql/dml/copy) | `COPY` -- refused by newer deployment versions | | [Refresh](/sql/dml/refresh) | filling computed columns and materialized views | ### DQL {#dql} @@ -141,9 +141,10 @@ insert -- see [What SQL cannot do](#cannot). | | | | --- | --- | -| Vector search | no SQL form -- a query-builder operation | +| Vector search | [depends on the deployment](/basics/sql#no-vector-sql); the query builder always has it | | Merge insert | no SQL form -- needs a source relation and a per-match policy | | `information_schema` | not enabled, so `SHOW TABLES` and `SHOW COLUMNS` fail | +| `COPY` | refused by newer deployment versions -- see [Copy](/sql/dml/copy) | | `PREPARE` / `EXECUTE` | preparation is a [protocol feature](/sql/prepared-statements), not a statement | | Run on a local database | no engine to plan the statement | diff --git a/docs/web/sql/operators.mdx b/docs/web/sql/operators.mdx index 05cad2467..5ffecc9bf 100644 --- a/docs/web/sql/operators.mdx +++ b/docs/web/sql/operators.mdx @@ -44,8 +44,8 @@ IN`, which is the usual place this bites. See Arithmetic over a vector column is **list** arithmetic, not scalar arithmetic — a vector is a `FixedSizeList`. When the operation you want is a -similarity comparison, use [vector search](/search/vector-search); there is no -SQL form for it. +similarity comparison, use [vector search](/search/vector-search); whether SQL +can express it [depends on the deployment](/basics/sql#no-vector-sql). ## Literals {#literals}