mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-14 15:52:26 +00:00
Jack Ye
fe651ae003
The function catalogue is now real rather than a pointer. Scalar functions are listed by taxonomy -- math, string, unicode, regex, date/time, array, map, struct, crypto, type -- and the names come from what datafusion-functions 54 actually registers, not from what DataFusion documents on main. Aggregate and window pages gain the same treatment. Two things the audit corrected rather than copied. SHOW TABLES does not work. DataFusion implements it over information_schema, which this session does not enable, so it fails at plan time. It was documented as working here, on the SHOW page and in three Manage tables; all four are fixed, and the SQL overview now lists the absence alongside SHOW COLUMNS. Prepared statements are a Flight SQL protocol feature, not a PREPARE statement. A script carrying PREPARE ... AS SELECT will not run, while the same intent expressed through a client will. The new page says so rather than copying DataFusion's syntax. Every section of the Documentation tab that has a SQL counterpart now links to it -- 27 of 76 non-SQL pages, each pointing at the specific statement rather than at the reference index.
140 lines
5.8 KiB
Plaintext
140 lines
5.8 KiB
Plaintext
---
|
|
title: "SDKs and SQL"
|
|
sidebarTitle: "SDKs & SQL"
|
|
description: "The ways to reach LanceDB — the language SDKs, the SQL endpoint, the REST API — and what each can do."
|
|
icon: code-compare
|
|
keywords: ["python", "typescript", "rust", "java", "sql", "rest", "api reference", "sdk"]
|
|
---
|
|
|
|
The same table can be reached several ways: a language SDK, SQL, or the REST
|
|
API. They are views onto one engine, not separate products -- a table written
|
|
from Python is queried from SQL without an export step.
|
|
|
|
They are not interchangeable, though, and the differences are worth knowing
|
|
before you pick one.
|
|
|
|
## What each one can do {#matrix}
|
|
|
|
| | Python | TypeScript | SQL |
|
|
| --- | :---: | :---: | :---: |
|
|
| Create and manage catalog objects | yes | yes | partly |
|
|
| Insert, merge, update, delete | yes | yes | yes |
|
|
| Filter, project, aggregate, join | yes | yes | yes |
|
|
| Vector search | yes | yes | -- |
|
|
| Full-text search | yes | yes | yes |
|
|
| Feature engineering, jobs | yes | partly | yes |
|
|
| Runs against a local database | yes | yes | **no** |
|
|
|
|
Three entries in that table do most of the work of explaining the rest.
|
|
|
|
## SQL needs a deployment {#sql-is-remote}
|
|
|
|
<Warning>
|
|
SQL is not available on a local database. A connection to a directory or a
|
|
bucket has no server to plan and execute a statement, and asking it to raises
|
|
`NotImplementedError`.
|
|
</Warning>
|
|
|
|
```python
|
|
db = lancedb.connect("./.lancedb") # no SQL
|
|
db = lancedb.connect("db://my-database") # SQL
|
|
```
|
|
|
|
This is the largest behavioural difference between the embedded and deployed
|
|
forms, and the reason the SDKs are the portable choice: SDK code moves between
|
|
the two by changing the URI, SQL does not.
|
|
|
|
<Badge color="red">Enterprise</Badge> Queries are served over an
|
|
[Arrow Flight SQL](https://arrow.apache.org/docs/format/FlightSql.html)
|
|
endpoint, so any Flight SQL client can reach it and results arrive as Arrow
|
|
record batches.
|
|
|
|
## Vector search has no SQL form {#no-vector-sql}
|
|
|
|
Similarity search is a query-builder operation. A query that needs both vector
|
|
similarity and SQL analytics runs the search in an SDK and the analytics over
|
|
its result. Full-text search *is* available in SQL, as the
|
|
[`fts()`](/sql/functions#fts) table function and the
|
|
[`contains_tokens`](/sql/scalar-functions#contains-tokens) predicate.
|
|
|
|
## TypeScript has no SQL method {#ts-no-sql}
|
|
|
|
The TypeScript SDK does not expose SQL, and has no methods for
|
|
[functions](/management/function). Reach the endpoint with a Flight
|
|
SQL client instead -- see [connecting a client](/search/sql). This is a gap in
|
|
the SDK rather than in the endpoint: the same statements work.
|
|
|
|
## Running SQL from Python {#sql-from-python}
|
|
|
|
```python
|
|
reader = db.execute_query("SELECT id, text FROM my_docs LIMIT 10")
|
|
table = reader.read_all()
|
|
```
|
|
|
|
`execute_query` returns a blocking Arrow reader once results start arriving; it
|
|
does not wait for the whole query. `execute_query_async` returns a handle
|
|
instead -- see [Query and job](/basics/jobs#queries).
|
|
|
|
## Choosing between them {#choosing}
|
|
|
|
Reach for **the SDKs** when you are writing data, managing the catalog, running
|
|
vector search, or building an application that should work the same embedded and
|
|
deployed.
|
|
|
|
Reach for **SQL** when the question is analytical -- joins across tables,
|
|
aggregates, window functions -- or when the thing asking is a BI tool, a
|
|
notebook, or a person who already thinks in SQL. It is also the only way to
|
|
create a [view](/management/view).
|
|
|
|
The two compose. A common shape is SDK code to build and maintain tables, and
|
|
SQL for the analysis on top.
|
|
|
|
## Reference {#reference}
|
|
|
|
### Officially supported SDKs {#supported-sdks}
|
|
|
|
Python, TypeScript and Rust are supported against both embedded and deployed
|
|
LanceDB.
|
|
|
|
| Reference | Description |
|
|
|:--------------|-------------------|
|
|
| <Icon icon="python" /> [Python](https://lancedb.github.io/lancedb/python/python/) | Full-featured Python client with pandas and numpy integration |
|
|
| <Icon icon="square-js" /> [TypeScript](https://lancedb.github.io/lancedb/js/) | A TypeScript wrapper around the Rust library, built with `napi-rs` |
|
|
| <Icon icon="rust" /> [Rust](https://docs.rs/lancedb/latest/lancedb/index.html) | Native Rust library with persistent storage and high performance |
|
|
|
|
### REST API {#rest}
|
|
|
|
<Badge color="red">Enterprise</Badge> A deployment implements the
|
|
[Lance REST Namespace](/basics/lance#enterprise), so the catalog is reachable
|
|
over HTTP by anything that speaks it -- see the
|
|
[REST API reference](/api-reference/rest).
|
|
|
|
| Reference | Description |
|
|
|:--------------|-------------------|
|
|
| <Icon icon="java" /> [Java](https://lancedb.github.io/lancedb/java/java/) | REST API Enterprise SDK in Java |
|
|
|
|
### Community SDKs {#community-sdks}
|
|
|
|
Contributed by the community rather than supported by LanceDB, so feature parity
|
|
and release cadence vary.
|
|
|
|
| Reference | Description |
|
|
|:--------------|-------------------|
|
|
| <Icon icon="golang" /> [Go](https://pkg.go.dev/github.com/lancedb/lancedb-go/pkg/lancedb) | Community-contributed Go SDK |
|
|
| <Icon icon="gem" /> [Ruby](https://github.com/scientist-labs/lancelot) | Community-contributed Ruby bindings |
|
|
| <Icon icon="swift" /> [Swift](https://github.com/RyanLisse/LanceDbSwiftKit) | Community-contributed Swift SDK |
|
|
| <Icon icon="R" /> [R](https://github.com/CathalByrneGit/lancedb) | Community-contributed R package |
|
|
| <Icon icon="flutter" /> [Flutter](https://github.com/Alexcn/flutter_lancedb) | Community-contributed Flutter bindings |
|
|
|
|
## What stays the same {#invariants}
|
|
|
|
Whichever you pick:
|
|
|
|
- **Types are Arrow types.** A vector column is a `FixedSizeList<Float32>`
|
|
everywhere, not a SQL type mapped onto storage. See
|
|
[Functions and types](/sql/functions).
|
|
- **Reads are Arrow.** Nothing is serialized into language objects on the way out
|
|
unless you convert it.
|
|
- **The catalog is one catalog.** A table created from TypeScript is visible to
|
|
Python and to SQL immediately -- there is no per-language namespace.
|