rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-10-26 05:26:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge main into gatekeeper/fix-2325-1

Browse Source
This commit is contained in:
Gatefixer
2026-10-25 09:50:49 +00:00
parent 82323dfba9 dcfef7c5ab
commit 10541cc2d8
141 changed files with 8196 additions and 1464 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/src/js/classes/Connection.md
+17 -63
View File
@@ -653,94 +653,48 @@ See [Listing](Listing.md) for pagination state and error behavior.
### listNamespaces()
```ts
abstract listNamespaces(namespacePath?, options?): Promise<ListNamespacesResponse>
abstract listNamespaces(namespacePath?, options?): Listing<string>
```
List the immediate child namespaces under the given parent.
Results may be paginated. To retrieve subsequent pages, pass the
`pageToken` returned by a previous call.
Lazily list immediate child namespaces, 100 per page by default.
#### Parameters
* **namespacePath?**: `string`[]
The parent namespace path. Defaults
to the root namespace if omitted.
* **options?**: `Partial`&lt;[`ListNamespacesOptions`](../interfaces/ListNamespacesOptions.md)&gt;
Pagination options
(`pageToken`, `limit`).
* **options?**: [`ListingOptions`](../interfaces/ListingOptions.md)
#### Returns
`Promise`&lt;[`ListNamespacesResponse`](../interfaces/ListNamespacesResponse.md)&gt;
Child namespace names and
an optional token for fetching the next page.
[`Listing`](Listing.md)&lt;`string`&gt;
***
### listTables()
#### listTables(options)
```ts
abstract listTables(options?): Promise<ListTablesResponse>
abstract listTables(namespacePath?, options?): Listing<string>
```
List a page of the tables in this database.
Lazily list tables in a namespace, requesting 100 per page by default.
To retrieve the tables after the page, pass the `pageToken` the response
carries back in. A page can be shorter than `limit` without being the last
one, so walk until a response carries no page token:
```ts
const names = [];
let pageToken = undefined;
do {
const page = await conn.listTables({ pageToken, limit: 100 });
names.push(...page.tables);
pageToken = page.pageToken;
} while (pageToken);
```
##### Parameters
* **options?**: `Partial`&lt;[`ListTablesOptions`](../interfaces/ListTablesOptions.md)&gt;
Pagination options
(`pageToken`, `limit`).
##### Returns
`Promise`&lt;[`ListTablesResponse`](../interfaces/ListTablesResponse.md)&gt;
A page of table names and an
optional token for the tables after it.
#### listTables(namespacePath, options)
```ts
abstract listTables(namespacePath?, options?): Promise<ListTablesResponse>
```
List a page of the tables in this database.
##### Parameters
#### Parameters
* **namespacePath?**: `string`[]
The namespace path to list tables from
(defaults to root namespace)
* **options?**: `Partial`&lt;[`ListTablesOptions`](../interfaces/ListTablesOptions.md)&gt;
Pagination options
(`pageToken`, `limit`).
* **options?**: [`ListingOptions`](../interfaces/ListingOptions.md)
##### Returns
#### Returns
`Promise`&lt;[`ListTablesResponse`](../interfaces/ListTablesResponse.md)&gt;
[`Listing`](Listing.md)&lt;`string`&gt;
A page of table names and an
optional token for the tables after it.
#### Example
```ts
for await (const name of conn.listTables([], { pageLimit: 20 })) {
console.log(name);
}
```
***
docs/src/js/classes/Job.md
+8 -5
View File
@@ -17,7 +17,8 @@ extra round trip on every call:
- [Job.refresh](Job.md#refresh) and [Job.status](Job.md#status) fetch the whole record.
- [Job.wait](Job.md#wait) records the terminal state it establishes, but not the
rest of the record.
- Everything is null until one of those runs.
- Registry details are null until one of those runs. Query handles report
jobType "query" immediately.
## Accessors
@@ -57,8 +58,10 @@ get id(): null | string
Identifies the operation on the server that is running it.
Operations that run in this process have no server id. The value is
opaque: parsing it or storing it to resume the job later is not supported.
Registry handles expose a registry job ID; query handles expose a
connection-scoped SQL query UUID string. Local handles have no ID.
Query handles report jobType "query". Their IDs are not registry IDs or
durable resume tokens.
#### Returns
@@ -72,8 +75,8 @@ opaque: parsing it or storing it to resume the job later is not supported.
get jobType(): null | string
```
The job's type, as the server names it. Null for an in-process job, which
has no server-side record.
"query" immediately for SQL-backed handles; otherwise the cached
server-reported job type. Null for in-process handles.
#### Returns
docs/src/js/classes/Table.md
+5 -1
View File
@@ -1298,7 +1298,11 @@ Specify exactly one of `column` or `indexName`.
Model-backed tokenizers such as `jieba/*` and `lindera/*` are rebuilt in
the client process from index metadata. For remote tables, this means the
same tokenizer model files must also exist locally.
same tokenizer model files must also exist locally (under
`LANCE_LANGUAGE_MODEL_HOME`) and must match the files the server uses. A
mismatch cannot be detected: the returned tokens may differ from what the
server's index produces. If the model files are missing, an error naming
the tokenizer is raised.
#### Parameters
docs/src/js/globals.md
-4
View File
@@ -108,10 +108,6 @@
- [JobEventsOptions](interfaces/JobEventsOptions.md)
- [JobFailureInfo](interfaces/JobFailureInfo.md)
- [JobInfo](interfaces/JobInfo.md)
- [ListNamespacesOptions](interfaces/ListNamespacesOptions.md)
- [ListNamespacesResponse](interfaces/ListNamespacesResponse.md)
- [ListTablesOptions](interfaces/ListTablesOptions.md)
- [ListTablesResponse](interfaces/ListTablesResponse.md)
- [ListingOptions](interfaces/ListingOptions.md)
- [MaterializedViewDefinition](interfaces/MaterializedViewDefinition.md)
- [MemtableStats](interfaces/MemtableStats.md)
docs/src/js/interfaces/CreateTableOptions.md
+4 -1
View File
@@ -65,7 +65,7 @@ then no error will be raised.
### mode
```ts
mode: "overwrite" | "create";
mode: "overwrite" | "create" | "exist_ok";
```
The mode to use when creating the table.
@@ -76,6 +76,9 @@ happen. Any provided data will be ignored.
If this is set to "overwrite" then any existing table will be replaced.
If this is set to "exist_ok" then an existing table is opened and any
provided data is ignored.
***
### schema?
docs/src/js/interfaces/ListNamespacesOptions.md
-27
View File
@@ -1,27 +0,0 @@
[**@lancedb/lancedb**](../README.md) • **Docs**
***
[@lancedb/lancedb](../globals.md) / ListNamespacesOptions
# Interface: ListNamespacesOptions
## Properties
### limit?
```ts
optional limit: number;
```
An optional limit to the number of results to return.
***
### pageToken?
```ts
optional pageToken: string;
```
Token from a previous response for pagination.
docs/src/js/interfaces/ListNamespacesResponse.md
-23
View File
@@ -1,23 +0,0 @@
[**@lancedb/lancedb**](../README.md) • **Docs**
***
[@lancedb/lancedb](../globals.md) / ListNamespacesResponse
# Interface: ListNamespacesResponse
## Properties
### namespaces
```ts
namespaces: string[];
```
***
### pageToken?
```ts
optional pageToken: string;
```
docs/src/js/interfaces/ListTablesOptions.md
-35
View File
@@ -1,35 +0,0 @@
[**@lancedb/lancedb**](../README.md) • **Docs**
***
[@lancedb/lancedb](../globals.md) / ListTablesOptions
# Interface: ListTablesOptions
## Properties
### limit?
```ts
optional limit: number;
```
An upper bound on how many tables to return.
Defaults to 100. Zero returns an empty page without a continuation token.
A page may hold fewer than this and still not be the last one, so keep
going while the response carries a page token rather than while pages are
full.
***
### pageToken?
```ts
optional pageToken: string;
```
Token from a previous response, to resume listing where it left off.
The token is opaque: it carries whatever the database needs to resume, and
callers should not construct or interpret one.
docs/src/js/interfaces/ListTablesResponse.md
-23
View File
@@ -1,23 +0,0 @@
[**@lancedb/lancedb**](../README.md) • **Docs**
***
[@lancedb/lancedb](../globals.md) / ListTablesResponse
# Interface: ListTablesResponse
## Properties
### pageToken?
```ts
optional pageToken: string;
```
***
### tables
```ts
tables: string[];
```
docs/src/js/interfaces/QueryExecutionOptions.md
+5 -3
View File
@@ -20,12 +20,14 @@ How to return blob v2 columns. Applies to `toArrow()` and `toArray()`.
- `"descriptions"` (default): each blob is a descriptor. Read the bytes
with [Table.fetchBlobs](../classes/Table.md#fetchblobs) or [Table.fetchBlobFiles](../classes/Table.md#fetchblobfiles).
- `"bytes"`: each top-level blob column holds the blob bytes as
`LargeBinary`. The query also reads `_rowid` to fetch the bytes, and
- `"bytes"`: each blob column, including blobs nested in structs, holds
the bytes as `LargeBinary`. The query also reads `_rowid` to fetch them, and
leaves it out of the result unless [QueryBase.withRowId](../classes/QueryBase.md#withrowid) was
called. All the bytes are held in memory.
Blobs nested in a struct or a list keep their descriptors.
Selecting a parent struct or directly aliasing a blob or struct also returns
bytes. Blobs inside lists keep their descriptors. Arbitrary projection
expressions are not materialized as bytes.
The rows and the bytes come from the same table version. If the table
changes while the query runs, the query runs again, up to three times.
docs/src/js/interfaces/TableNamesOptions.md
+1 -1
View File
@@ -8,7 +8,7 @@
## Deprecated
Use [ListTablesOptions](ListTablesOptions.md) with [Connection.listTables](../classes/Connection.md#listtables)
Use [ListingOptions](ListingOptions.md) with [Connection.listTables](../classes/Connection.md#listtables)
instead.
## Properties
docs/src/python/python.md
+90 -8
View File
@@ -26,14 +26,17 @@ is also an [asynchronous API client](#connections-asynchronous).
::: lancedb.db.DBConnection
Resource listings (`list_secrets`, `list_views`, `list_jobs`, `list_functions`, and
`list_materialized_views`) return lazy iterators. Use `list(db.list_views())` to
collect synchronous results, or `[name async for name in db.list_views()]` with
an asynchronous connection. `page_limit` controls each request, and `page_token`
resumes a listing. Drain the iterator's cached results before saving its token.
Resource listings return iterators. Use `list(db.list_tables())` to collect all
table names. `db.list_tables().tables` is a list of the first page; `limit`
controls its page size and `page_token` resumes at a saved token. The same
patterns apply to `list_namespaces()`. Other resource listings use `page_limit`.
::: lancedb.listing.Listing
::: lancedb.listing.TableListing
::: lancedb.listing.NamespaceListing
::: lancedb.Session
## Catalogs (Synchronous)
@@ -218,10 +221,19 @@ The connected database and `default_namespace_path=["public"]` are used for
unqualified tables. Fully qualified references can still query other databases
and namespaces available to the same deployment. `execute_query` returns a
reader as soon as its initial result stream is available. `execute_query_async`
returns a query handle immediately; use it to inspect progress, open a reader,
returns a query handle once submission is accepted; use it to inspect progress, open a reader,
or cancel the query. The SQL client is initialized by the first query and
retained for the lifetime of the remote connection. Query ids are random,
connection-scoped references rather than encoded SQL or durable resume tokens:
connection-scoped references rather than encoded SQL or durable resume tokens.
Both APIs start with one Flight call. The server can stream results on that
call or return a polling continuation that the SDK follows automatically.
This behavior is the same with and without parameters. Open the reader directly;
a directly streamed query needs its reader to make progress. Dropping an unread
handle or its reader stops direct streaming. Deferred execution continues until
explicitly cancelled.
For example:
```python
import lancedb
@@ -303,6 +315,14 @@ listing a storage directory.
## Functions and Jobs
Annotate a blob input with `lancedb.BlobReader` for lazy, seekable access, or
with `bytes` to materialize its contents before the Function runs. Reader
inputs infer the Blob V2 type without an explicit input schema. Readers are
scoped to each invocation and support `read`, `readinto`, `seek`, `tell`,
`size`, and cursor-independent `read_range`.
::: lancedb.BlobReader
::: lancedb.functions.FunctionArtifact
::: lancedb.functions.FunctionParameter
@@ -520,12 +540,64 @@ instead of being materialized with the rest of the row.
`lancedb.BlobType` is `lance.blob.BlobType` when pylance is installed. Without
pylance, LanceDB uses a matching `lance.blob.v2` extension type so blob columns
still work. Queries return descriptors. Call
still work. Native table queries return descriptors. Call
[`fetch_blob_files`][lancedb.table.Table.fetch_blob_files] for lazy reads or
[`fetch_blobs`][lancedb.table.Table.fetch_blobs] for eager bytes.
One missing or deleted row ID rejects the entire blob-read batch; the APIs do
not return partial results.
Remote SQL returns blob bytes by default. Select `blob_descriptor(video)` to
return an openable reference instead, then use
[`open_blob_file`][lancedb.table.Table.open_blob_file] or the lazy iterable
[`open_blob_files`][lancedb.table.Table.open_blob_files]:
```python
rows = db.execute_query("""
SELECT f.timestamp, blob_descriptor(v.video) AS video
FROM videos v JOIN frames f ON v.id = f.video_id
""").read_all()
videos = db.open_table("videos")
for row in rows.to_pylist():
if row["video"] is not None:
with videos.open_blob_file(row["video"]) as video:
frame = extract_frame(video, row["timestamp"])
```
This example uses a remote SQL connection that produces source-bearing
descriptors. The descriptor contains `kind`, `position`, `size`, `blob_id`,
`blob_uri`, and a `source` struct with `uri`, `version`, `committed_at_micros`,
`branch`, `column`, `row_reference`, and `row_is_address`. The opener verifies
the source table and commit and reads the captured version without changing
the table handle's checkout. Raw native storage descriptors omit `source`;
use `fetch_blob_files(column, row_ids)` for those query results.
The singular API accepts a dictionary or Arrow struct scalar. The iterable
accepts Arrow arrays/chunked arrays or another iterable, opens handles as
iteration advances, and preserves order, duplicates, and nulls as `None`.
Close each non-null handle after use. Async tables provide
`await table.open_blob_file(...)` and
`async for file in table.open_blob_files(...)`.
```python
for row, video in zip(rows.to_pylist(), videos.open_blob_files(rows["video"])):
if video is not None:
with video:
frame = extract_frame(video, row["timestamp"])
```
For an async table, the iterable is consumed with `async for`:
```python
async for video in async_videos.open_blob_files(rows["video"]):
if video is not None:
with video:
payload = await video.aread()
```
Payload bytes load on reads. Opening an external blob with unspecified size
can issue a metadata-only HEAD request. Remote reads use authenticated range
requests that validate the captured URI and commit timestamp.
Blobs inside lists can be written and queried as descriptors. They are omitted
from `blob_columns()` because row ids do not identify individual list elements,
so the fetch APIs cannot read their bytes yet.
@@ -633,6 +705,16 @@ can be used to create, list, or open tables.
::: lancedb.listing.AsyncListing
`async for name in db.list_tables()` loads pages as needed. To inspect the
first page, use `page = await db.list_tables()` and then `page.tables`. Reading
the page list does not consume the iterator. `page.page_token` is the token for
the next request; drain cached results before using it to resume in a new
listing. The same behavior applies to namespaces.
::: lancedb.listing.AsyncTableListing
::: lancedb.listing.AsyncNamespaceListing
## Namespaces (Asynchronous)
::: lancedb.connect_namespace_async
docs/web-tests/ts/connection.test.ts
+11 -5
View File
@@ -4,6 +4,12 @@ import { afterEach, describe, expect, jest, test } from "@jest/globals";
import * as lancedb from "@lancedb/lancedb";
import { withTempDirectory } from "./util.ts";
async function collectNames(items: AsyncIterable<string>): Promise<string[]> {
const names: string[] = [];
for await (const name of items) names.push(name);
return names;
}
test("connect to a local database", async () => {
await withTempDirectory(async (tempDir) => {
const cwd = process.cwd();
@@ -13,12 +19,12 @@ test("connect to a local database", async () => {
const db = await lancedb.connect("ex_lancedb");
// --8<-- [end:connect]
await db.createTable("probe", [{ id: 1 }]);
expect((await db.listTables()).tables).toEqual(["probe"]);
expect(await collectNames(db.listTables())).toEqual(["probe"]);
} finally {
process.chdir(cwd);
}
const reopened = await lancedb.connect(`${tempDir}/ex_lancedb`);
expect((await reopened.listTables()).tables).toEqual(["probe"]);
expect(await collectNames(reopened.listTables())).toEqual(["probe"]);
});
});
@@ -109,8 +115,8 @@ async function namespaceTableOpsExample() {
);
// Verify
console.log((await db.listNamespaces()).namespaces); // ["prod"]
console.log((await db.listNamespaces(["prod"])).namespaces); // ["recommendations", "search"]
console.log(await collectNames(db.listNamespaces())); // ["prod"]
console.log(await collectNames(db.listNamespaces(["prod"]))); // ["recommendations", "search"]
console.log(await db.tableNames(["prod", "search"])); // ["user"]
console.log(await db.tableNames(["prod", "recommendations"])); // ["user"]
// --8<-- [end:namespace_table_ops]
@@ -124,7 +130,7 @@ async function namespaceAdminOpsExample() {
await db.createNamespace(["prod"]);
await db.createNamespace(["prod", "search"]);
const childNamespaces = (await db.listNamespaces(["prod"])).namespaces;
const childNamespaces = await collectNames(db.listNamespaces(["prod"]));
console.log(`Child namespaces under ${JSON.stringify(namespace)}:`, childNamespaces);
// Child namespaces under ["prod","search"]: [ 'search' ]
docs/web-tests/ts/quickstart.test.ts
+1 -6
View File
@@ -20,12 +20,7 @@ const connections =
async function tableNames(db: lancedb.Connection) {
const names: string[] = [];
let pageToken: string | undefined;
do {
const page = await db.listTables({ pageToken });
names.push(...page.tables);
pageToken = page.pageToken;
} while (pageToken);
for await (const name of db.listTables()) names.push(name);
return names;
}
docs/web/basics/jobs.mdx
+31 -6
View File
@@ -20,19 +20,20 @@ failures.
Methods such as Python's `create_index_async` and TypeScript's
`createIndexAsync` return job handles. In OSS, the work runs in your application
process; in Enterprise, it runs on server workers. A returned handle can
represent an operation that has already finished, so check its status before
process; in Enterprise, it runs on server workers or through the SQL query
backend. A returned handle can represent an operation that has already finished,
so check its status before
waiting or requesting cancellation. See the operation's API reference for when
the method returns.
## Submit and wait for a job {#waiting}
1. **Submit the operation.** Keep the returned job handle. For an Enterprise
job, also keep its ID if another process needs to look it up later.
registry job, also keep its ID if another process needs to look it up later.
2. **Check progress.** Read the current status while the operation runs.
3. **Wait for completion.** Check that the job succeeded before using its
result. If it failed or was cancelled, inspect the failure details and any
results it committed before stopping.
result. If it failed or was cancelled, inspect the failure details and any
results it committed before stopping.
In Python, `job.status()` reports the current state and `job.wait()` waits for
completion, raising if the job fails or is cancelled. If `job.wait()` times out, the client stops waiting.
@@ -59,13 +60,37 @@ history.
## OSS and Enterprise execution {#in-process}
| | OSS | Enterprise |
The following table describes local operations and Enterprise registry jobs.
SQL-backed job handles follow the query lifecycle described below.
| | OSS | Enterprise registry job |
| --- | --- | --- |
| Execution | In your application process | On server workers |
| Job identity | No server ID | A server ID for looking up the job |
| Event history | No server event history | Available through the jobs API |
| Reattaching after a client restart | No server job to reopen | Reopen by ID while the record is retained |
### SQL-backed job handles
Remote materialized-view CREATE and REFRESH use the SQL query backend. Python's
`create_materialized_view_async()` and `mv.refresh_async()` still return job
handles with `status()`, `wait()` and `cancel()`. Synchronous creation and refresh
wait on the same SQL execution before returning.
The existing `job.job_type` identifies query-backed handles as `"query"` from
submission, and `job.id` contains the connection-scoped SQL query UUID string.
Use `describe_query(job.id)` on the submitting connection to inspect the query.
Registry handles retain their server-reported job types (available after
`status()` or `refresh()`) and registry IDs. Local handles have no ID or job type.
SQL-backed handles do not expose registry records or event history and cannot
be reopened through `open_job()` or reattached after a client restart. Their IDs
are not durable resume tokens. Cancellation follows the SQL server's
capabilities and can be unsupported.
The server decides whether SQL executes on its serving fleet or starts SQL job
workers. That placement does not change the handle's query-backed identity.
## Learn more {#next}
- [Build and repair indexes](/indexing/reindexing): submit index work.
docs/web/basics/sql.mdx
+2
View File
@@ -82,6 +82,8 @@ In application code, load credentials from environment variables or a secrets
manager. The [connection guide](/basics/connect#enterprise) shows that setup.
`execute_query()` returns a blocking Arrow reader when results start arriving.
The server can stream rows on the first call or return result tickets that the
SDK follows automatically, with the same behavior for parameterized queries.
The example uses `read_all()` to collect the small result in memory. For larger
results, consume the reader in batches. To submit a query and check its status
separately, use `execute_query_async()`. See [Queries](/basics/query) for query
docs/web/docs.json
+1
View File
@@ -350,6 +350,7 @@
"pages": [
"enterprise/index",
"enterprise/architecture",
"enterprise/distributed-search",
"enterprise/deployment/index",
"deployment/storage",
"deployment/monitoring",
docs/web/enterprise/architecture.mdx
+3
View File
@@ -18,3 +18,6 @@ run on machines that can be replaced without any of them owning the data.
[Architecture](/architecture) covers this alongside the embedded form, which is
the comparison most readers want first.
See [Distributed search](/enterprise/distributed-search) for how Enterprise
distributes index execution and table-data reads.
docs/web/enterprise/distributed-search.mdx
+33
View File
@@ -0,0 +1,33 @@
---
title: "Distributed search"
sidebarTitle: "Distributed search"
description: "How LanceDB Enterprise distributes index execution and table-data reads."
icon: "network-wired"
availability:
oss: unavailable
enterprise: available
summary: >-
LanceDB Enterprise distributes index execution and table-data reads across
plan executors. Embedded LanceDB executes searches in the application process.
---
Distributed search spreads index execution, table-data reads, and caching across
multiple serving machines in LanceDB Enterprise. Query nodes plan queries and
combine results. Plan executors cache index structures and table data and execute
the remote work. See [Enterprise architecture](/enterprise/architecture).
## Embedded search {#embedded-search}
LanceDB OSS runs inside your application process. It searches indexes and reads
table data using that process's resources, including its index cache. Reading a
table from object storage does not by itself distribute query execution across
machines.
## Search interfaces {#search-interfaces}
The shared search guides describe the query APIs and index settings:
- [Vector search](/search/vector-search) and [vector indexes](/indexing/vector-index).
- [Full-text search](/search/full-text-search) and [FTS indexes](/indexing/fts-index).
- [Filtering](/search/filtering) and [scalar indexes](/indexing/scalar-index).
- [Query plans](/search/explain) for inspecting execution and runtime measurements.
docs/web/indexing/vector-index.mdx
+1 -1
View File
@@ -384,7 +384,7 @@ The inner workings of RaBitQ quantization are rather mathematically dense. It ge
applying a uniformly random, approximately distance-preserving orthogonal transformation of the vertices of the $d$-dimensional hypercube,
where $d$ is the dimensionality of the dataset. We defer the details, and an elegant theoretical error bound, to [the original paper.](https://arxiv.org/pdf/2405.12497)
##### Using RaBitQ
##### Using RaBitQ {#using-rabitq}
Use RaBitQ quantization by selecting quantized index types ending in the suffix `RQ`. For example, call `create_index` with `index_type="IVF_RQ"`.
Note that when using `IVF_RQ`, the dimension of the dataset must be a multiple of `8`.
docs/web/integrations/embedding/ollama.mdx
+3
View File
@@ -10,6 +10,9 @@ Generate embeddings via the [ollama](https://github.com/ollama/ollama-python) py
- [Ollama docs on embeddings](https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings)
- [Ollama blog on embeddings](https://ollama.com/blog/embedding-models)
For direct calls, `compute_source_embeddings` accepts a list of strings or a
one-dimensional NumPy string array. Both inputs are sent to Ollama in one batch.
| Parameter | Type | Default Value | Description |
|------------------------|----------------------------|--------------------------|------------------------------------------------------------------------------------------------------------------------------------------------|
| `name` | `str` | `nomic-embed-text` | The name of the model. |
docs/web/integrations/embedding/voyageai.mdx
+9 -2
View File
@@ -45,6 +45,8 @@ Supported parameters (to be passed in `create` method) are:
| `output_dimension` | `int` | `None` | Output embedding dimension. Only supported by `voyage-multimodal-3.5`. Valid options: 256, 512, 1024 (default), 2048. |
For text models, `compute_source_embeddings()` accepts a single string, a list of strings, a one-dimensional NumPy array, or a PyArrow `Array` or `ChunkedArray`. Batch inputs retain their order.
Usage Example:
<CodeGroup>
@@ -55,10 +57,15 @@ Usage Example:
### Multimodal Example {#multimodal-example}
The `voyage-multimodal-3.5` model can embed text alongside images. You can use image URLs, file paths, or PIL Image objects:
The `voyage-multimodal-3.5` model can embed text alongside images. You can use image URLs, local `pathlib.Path` objects, raw image bytes, or PIL Image objects.
For source embeddings, PIL images and images loaded from bytes or local paths are
converted to RGB and encoded as JPEG. This supports RGBA, grayscale with alpha,
and palette images; transparency is discarded. Composite transparent images onto
your chosen background before embedding if the background color matters.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{PyEmbeddingVoyageaiMultimodal}
</CodeBlock>
</CodeGroup>
</CodeGroup>
docs/web/integrations/reranking/openai.mdx
+1 -1
View File
@@ -26,7 +26,7 @@ credentials from environment variables or a secrets manager.
## Accepted Arguments {#accepted-arguments}
| Argument | Type | Default | Description |
| --- | --- | --- | --- |
| `model_name` | `str` | `"gpt-4-turbo-preview"` | The name of the reranker model to use.|
| `model_name` | `str` | `"gpt-5.6-sol"` | The name of the reranker model to use.|
| `column` | `str` | `"text"` | The name of the column to use as input to the cross encoder model. |
| `return_score` | `str` | `"relevance"` | Options are "relevance" or "all". The type of score to return. If "relevance", will return only the `_relevance_score. If "all" is supported, will return relevance score along with the vector and/or fts scores depending on query type. |
| `api_key` | `str` | `None` | The API key to use. If None, will use the OPENAI_API_KEY environment variable.
docs/web/management/function.mdx
+34
View File
@@ -205,6 +205,40 @@ Function output blobs follow the same compaction and cleanup lifecycle as
ingested Managed blobs. Retained versions continue to own their payloads;
cleanup reclaims an object only after its retained references are gone.
### Draft: incremental blob inputs (upcoming)
This feature requires an SDK that exports `BlobReader` and a compatible
Enterprise runtime with reader input support enabled. Coordinate the runtime
upgrade before registering or executing reader Functions.
Annotate a row Function parameter with `BlobReader` to read a Blob v2 input
incrementally. The signature selects the access mode without another
registration option. For example, read the first MP4 box type without loading
the whole video:
```python
from lancedb import BlobReader, udf
@udf
def first_box_type(video: BlobReader) -> bytes:
video.seek(4)
return video.read(4)
```
The worker passes a seekable reader backed by the stored blob. `read(size)`
reads from the current cursor; `read_range(offset, length)` reads a byte range
without moving it. `size()` returns the payload size without reading it.
Each row call receives its own reader, which closes when the call returns or
raises. Do not retain it for another invocation.
Use `video: bytes` when the Function needs the complete payload. Bytes inputs
materialize before invocation and remain subject to eager input limits.
Reader inputs keep the unread payload out of the input batch, but calling
`video.read()` without a size loads all remaining bytes in Python. Reader
inputs are supported in row mode. Return values remain ordinary values such
as `bytes`; this feature does not add streamed outputs.
## Declare the environment {#environment}
The function runs remotely, so its dependencies are part of its definition:
docs/web/quickstart.mdx
+27
View File
@@ -58,6 +58,33 @@ uv add --prerelease allow --index https://pypi.fury.io/lancedb/ lancedb
```
</CodeGroup>
If you also need the optional `lance` Python API, install the `pylance` extra and
include the Lance preview index:
<CodeGroup>
```bash Python (pip) icon="terminal"
pip install --pre \
--extra-index-url https://pypi.fury.io/lancedb/ \
--extra-index-url https://pypi.fury.io/lance-format/ \
"lancedb[pylance]"
```
```bash Python (uv) icon="terminal"
uv pip install --prerelease allow \
--index https://pypi.fury.io/lancedb/ \
--index https://pypi.fury.io/lance-format/ \
--index-strategy unsafe-best-match \
"lancedb[pylance]"
```
</CodeGroup>
The LanceDB preview index contains old `pylance` releases; current Lance previews
are published to the `lance-format` index. By default, uv only considers versions
from the first index containing a package, which can select an old `pylance` or
backtrack to an older LanceDB release. `unsafe-best-match` searches all configured
indexes, like pip; use it only with indexes you trust. Install the extra rather
than an unconstrained `pylance` so LanceDB's minimum supported version is enforced.
<Note>
Pre-release builds receive the same level of testing as stable releases, but their availability is not guaranteed
for more than 6 months after release. For real-world workloads, we recommend you use the latest stable release
docs/web/search/vector-search.mdx
+1 -1
View File
@@ -235,7 +235,7 @@ The table below summarizes the behavior of `_distance` in search results based o
See [vector indexes](/indexing/vector-index) for index configuration. The
following sections cover query-time settings.
### Tuning `approx_mode`
### Tuning `approx_mode` {#tuning-approx-mode}
Use `approx_mode` when you want to adjust the speed/recall trade-off for approximate vector search at query time. This setting currently applies only to RQ-quantized indexes, such as `IVF_RQ`; other index types ignore it.