mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-12 14:52:26 +00:00
docs: declare where SQL and the SDKs apply
SQL is unavailable in the embedded library and varies on a deployment, which serves it only when its Flight SQL endpoint is enabled and reachable, and the SDK has to be told where that is. The page's opening badge goes. The SDKs page declares both offerings. Neither SDK aggregates or joins, so the parity tables no longer say they do, and views can be created through the SDKs on a deployment. Pages that said vector search has no SQL form now point to the version-dependent SQL form, and COPY, which newer deployment versions refuse, is marked as such in the reference.
This commit is contained in:
@@ -83,8 +83,10 @@ return results. They are fast enough not to need a handle. See
|
||||
|
||||
Query-builder searches -- [vector](/search/vector-search),
|
||||
[full-text](/search/full-text-search), [hybrid](/search/hybrid-search) -- return
|
||||
results directly. They are fast enough not to need a handle, and they have no
|
||||
SQL form. See [Semantic Search](/search).
|
||||
results directly. They are fast enough not to need a handle. In SQL on a
|
||||
deployment, full-text search is the [`fts()`](/sql/functions#fts) table
|
||||
function and vector search [depends on the deployment](/basics/sql#no-vector-sql);
|
||||
hybrid search has no SQL form. See [Semantic Search](/search).
|
||||
|
||||
## When the work outlives the call {#jobs}
|
||||
|
||||
|
||||
+13
-5
@@ -4,6 +4,13 @@ sidebarTitle: "SDKs"
|
||||
description: "The language clients — Python, TypeScript, Rust — and what stays true across all of them."
|
||||
icon: code-compare
|
||||
keywords: ["python", "typescript", "rust", "java", "sdk", "rest", "api reference"]
|
||||
availability:
|
||||
oss: available
|
||||
enterprise: available
|
||||
summary: >-
|
||||
The Python, TypeScript and Rust SDKs work with embedded databases and,
|
||||
through `db://`, with Enterprise deployments; the Java SDK reaches
|
||||
deployments only. What each can do differs by language and connection.
|
||||
---
|
||||
|
||||
A language SDK is the primary way to work with LanceDB: create and manage
|
||||
@@ -29,14 +36,15 @@ Not every SDK covers the same surface, though:
|
||||
| --- | :---: | :---: |
|
||||
| Create and manage catalog objects | yes | yes |
|
||||
| Insert, merge, update, delete | yes | yes |
|
||||
| Filter, project, aggregate, join | yes | yes |
|
||||
| Filter, project | yes | yes |
|
||||
| Vector search | yes | yes |
|
||||
| Full-text search | yes | yes |
|
||||
| Feature engineering, jobs | yes | partly |
|
||||
| Runs against a local database | yes | yes |
|
||||
|
||||
TypeScript has no methods for [functions](/management/function). Reach the
|
||||
gap with SQL instead -- see [SQL](/basics/sql).
|
||||
Neither SDK aggregates or joins, and TypeScript has no methods for
|
||||
[functions](/management/function). Reach those with SQL on a deployment
|
||||
instead -- see [SQL](/basics/sql).
|
||||
|
||||
## REST API {#rest}
|
||||
|
||||
@@ -62,8 +70,8 @@ and release cadence vary.
|
||||
| <Icon icon="R" /> [R](https://github.com/CathalByrneGit/lancedb) | Community-contributed R package |
|
||||
| <Icon icon="flutter" /> [Flutter](https://github.com/Alexcn/flutter_lancedb) | Community-contributed Flutter bindings |
|
||||
|
||||
For any other language, [SQL](/basics/sql) reaches the same tables without a
|
||||
LanceDB-specific SDK at all.
|
||||
For any other language, [SQL](/basics/sql) on a deployment reaches the same
|
||||
tables without a LanceDB-specific SDK at all.
|
||||
|
||||
## Python multiprocessing {#multiprocessing}
|
||||
|
||||
|
||||
+21
-7
@@ -4,11 +4,14 @@ sidebarTitle: "SQL"
|
||||
description: "The same tables, reached with SQL — the universal access path, and where every operation is formally documented."
|
||||
icon: table-list
|
||||
keywords: ["sql", "flight sql", "datafusion", "reference", "dialect"]
|
||||
availability:
|
||||
oss: unavailable
|
||||
enterprise: varies
|
||||
summary: >-
|
||||
SQL runs on an Enterprise deployment whose Flight SQL endpoint is enabled
|
||||
and reachable; embedded connections raise instead.
|
||||
---
|
||||
|
||||
<Badge color="red">Enterprise</Badge> SQL runs on a deployment. A local
|
||||
database has no engine to plan a statement, so `execute_query` raises there.
|
||||
|
||||
SQL is a second way to reach the same tables the [SDKs](/basics/sdk) do --
|
||||
not a different product, a different interface onto one engine. A table
|
||||
written from Python is queried from SQL without an export step, and a table
|
||||
@@ -39,12 +42,17 @@ code speaks, the other is available too.
|
||||
| --- | :---: | :---: |
|
||||
| Create and manage catalog objects | yes | partly |
|
||||
| Insert, merge, update, delete | yes | yes |
|
||||
| Filter, project, aggregate, join | yes | yes |
|
||||
| Filter, project | yes | yes |
|
||||
| Aggregate, join | no | yes |
|
||||
| Vector search | yes | version-dependent |
|
||||
| Full-text search | yes | yes |
|
||||
| Feature engineering, jobs | yes | yes |
|
||||
| Runs against a local database | yes | **no** |
|
||||
|
||||
On a deployment, feature engineering must also be enabled by the deployment,
|
||||
whether it is declared in SQL or through an SDK -- see
|
||||
[Computed columns](/feature-engineering/add-computed-column).
|
||||
|
||||
## Vector search depends on the deployment {#no-vector-sql}
|
||||
|
||||
The SDK query builder provides vector search across embedded and deployed
|
||||
@@ -59,9 +67,9 @@ Full-text search is available through the [`fts()`](/sql/functions#fts) table
|
||||
function and the [`contains_tokens`](/sql/scalar-functions#contains-tokens)
|
||||
predicate.
|
||||
|
||||
The reverse gap also exists: a [view](/management/view) is a saved SQL query,
|
||||
so SQL is the only way to create one -- there's nothing for an SDK method to
|
||||
call.
|
||||
A [view](/management/view) is a saved SQL query. On a deployment you can
|
||||
create one with `CREATE VIEW` or by passing the query to an SDK's
|
||||
`create_view`; an embedded database has no views.
|
||||
|
||||
## SQL needs a deployment {#sql-is-remote}
|
||||
|
||||
@@ -76,6 +84,12 @@ db = lancedb.connect("./.lancedb") # no SQL
|
||||
db = lancedb.connect("db://my-database") # SQL
|
||||
```
|
||||
|
||||
A deployment serves SQL only when its Flight SQL endpoint is enabled and
|
||||
reachable, which is part of the deployment's configuration. The SDK also has to
|
||||
know where that endpoint is: pass `sql_host_override` when connecting, or
|
||||
connect through an `http://` `host_override`, from which the SDK derives it
|
||||
(the next port up, or 10025 when the override names no port).
|
||||
|
||||
## Running SQL from Python {#sql-from-python}
|
||||
|
||||
```python
|
||||
|
||||
@@ -6,7 +6,11 @@ icon: file-export
|
||||
keywords: ["copy", "export", "parquet", "sql"]
|
||||
---
|
||||
|
||||
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
|
||||
<Warning>
|
||||
Newer deployment versions refuse `COPY` over the SQL endpoint with "COPY is not
|
||||
supported". On those, export a result by reading it through a Flight SQL client
|
||||
or an SDK and writing the files from your application.
|
||||
</Warning>
|
||||
|
||||
```sql
|
||||
COPY (SELECT id, title FROM documents) TO 's3://bucket/export/' STORED AS PARQUET
|
||||
|
||||
@@ -112,7 +112,8 @@ estimate -- a 1% sample of a skewed column will miss rare categories entirely.
|
||||
|
||||
## What aggregation does not do {#not}
|
||||
|
||||
Aggregates run over rows the query already produced. They cannot combine with
|
||||
[vector search](/search/vector-search) in one statement, because similarity
|
||||
search has no SQL form. Run the search in an SDK and aggregate its result, or
|
||||
materialize the search output first.
|
||||
Aggregates run over rows the query already produced. Whether a statement can
|
||||
include [vector search](/search/vector-search)
|
||||
[depends on the deployment](/basics/sql#no-vector-sql); where it cannot, run the
|
||||
search in an SDK and aggregate its result, or materialize the search output
|
||||
first.
|
||||
|
||||
@@ -6,9 +6,9 @@ icon: table-list
|
||||
keywords: ["sql", "datafusion", "flight sql", "reference", "dialect"]
|
||||
---
|
||||
|
||||
<Badge color="red">Enterprise</Badge> SQL runs on a deployment. A local database
|
||||
has no engine to plan a statement, so `execute_query` raises there -- see
|
||||
[SQL](/basics/sql#sql-is-remote).
|
||||
<Badge color="red">Enterprise</Badge> SQL runs on a deployment that enables its
|
||||
Flight SQL endpoint. A local database has no SQL endpoint, so `execute_query`
|
||||
raises there -- see [SQL](/basics/sql#sql-is-remote).
|
||||
|
||||
This is a reference rather than a guide: one page per construct, syntax first.
|
||||
The guides elsewhere in these docs show the same operations from the SDKs.
|
||||
@@ -87,8 +87,8 @@ const result = await client.query("SELECT * FROM flights WHERE origin = 'SFO'");
|
||||
|
||||
The endpoint executes one statement per request and is meant for the queries
|
||||
and DDL/DML pages below. Reach for the SDKs instead for table-management
|
||||
operations that have no SQL form, such as building a vector index or a merge
|
||||
insert -- see [What SQL cannot do](#cannot).
|
||||
operations that have no SQL form, such as a merge insert -- see
|
||||
[What SQL cannot do](#cannot).
|
||||
|
||||
## Reference {#reference}
|
||||
|
||||
@@ -111,7 +111,7 @@ insert -- see [What SQL cannot do](#cannot).
|
||||
| [Insert](/sql/dml/insert) | `INSERT INTO`, including `INSERT ... SELECT` |
|
||||
| [Update](/sql/dml/update) | `UPDATE ... SET` |
|
||||
| [Delete](/sql/dml/delete) | `DELETE FROM` |
|
||||
| [Copy](/sql/dml/copy) | `COPY INTO` / `COPY TO` |
|
||||
| [Copy](/sql/dml/copy) | `COPY` -- refused by newer deployment versions |
|
||||
| [Refresh](/sql/dml/refresh) | filling computed columns and materialized views |
|
||||
|
||||
### DQL {#dql}
|
||||
@@ -141,9 +141,10 @@ insert -- see [What SQL cannot do](#cannot).
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| Vector search | no SQL form -- a query-builder operation |
|
||||
| Vector search | [depends on the deployment](/basics/sql#no-vector-sql); the query builder always has it |
|
||||
| Merge insert | no SQL form -- needs a source relation and a per-match policy |
|
||||
| `information_schema` | not enabled, so `SHOW TABLES` and `SHOW COLUMNS` fail |
|
||||
| `COPY` | refused by newer deployment versions -- see [Copy](/sql/dml/copy) |
|
||||
| `PREPARE` / `EXECUTE` | preparation is a [protocol feature](/sql/prepared-statements), not a statement |
|
||||
| Run on a local database | no engine to plan the statement |
|
||||
|
||||
|
||||
@@ -44,8 +44,8 @@ IN`, which is the usual place this bites. See
|
||||
<Warning>
|
||||
Arithmetic over a vector column is **list** arithmetic, not scalar arithmetic —
|
||||
a vector is a `FixedSizeList<Float32>`. When the operation you want is a
|
||||
similarity comparison, use [vector search](/search/vector-search); there is no
|
||||
SQL form for it.
|
||||
similarity comparison, use [vector search](/search/vector-search); whether SQL
|
||||
can express it [depends on the deployment](/basics/sql#no-vector-sql).
|
||||
</Warning>
|
||||
|
||||
## Literals {#literals}
|
||||
|
||||
Reference in New Issue
Block a user