mirror of
https://github.com/lancedb/lancedb.git
synced 2026-09-01 11:08:55 +00:00
8b7e13b0c6
In LanceDB Enterprise, we've adopted these conventions to give some "canonical" metadata paths. This lets us display them in a certain way in the UI or let agents standardize on them, to assume they'll find info in a certain place. This PR (only comments/docs) just documents those choices.
1382 lines
31 KiB
Markdown
1382 lines
31 KiB
Markdown
[**@lancedb/lancedb**](../README.md) • **Docs**
|
|
|
|
***
|
|
|
|
[@lancedb/lancedb](../globals.md) / Table
|
|
|
|
# Class: `abstract` Table
|
|
|
|
A Table is a collection of Records in a LanceDB Database.
|
|
|
|
A Table object is expected to be long lived and reused for multiple operations.
|
|
Table objects will cache a certain amount of index data in memory. This cache
|
|
will be freed when the Table is garbage collected. To eagerly free the cache you
|
|
can call the `close` method. Once the Table is closed, it cannot be used for any
|
|
further operations.
|
|
|
|
Tables are created using the methods [Connection#createTable](Connection.md#createtable)
|
|
and [Connection#createEmptyTable](Connection.md#createemptytable). Existing tables are opened
|
|
using [Connection#openTable](Connection.md#opentable).
|
|
|
|
Closing a table is optional. It not closed, it will be closed when it is garbage
|
|
collected.
|
|
|
|
## Accessors
|
|
|
|
### name
|
|
|
|
```ts
|
|
get abstract name(): string
|
|
```
|
|
|
|
Returns the name of the table
|
|
|
|
#### Returns
|
|
|
|
`string`
|
|
|
|
## Methods
|
|
|
|
### add()
|
|
|
|
```ts
|
|
abstract add(data, options?): Promise<AddResult>
|
|
```
|
|
|
|
Insert records into this Table.
|
|
|
|
#### Parameters
|
|
|
|
* **data**: [`Data`](../type-aliases/Data.md)
|
|
Records to be inserted into the Table
|
|
|
|
* **options?**: `Partial`<[`AddDataOptions`](../interfaces/AddDataOptions.md)>
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`AddResult`](../interfaces/AddResult.md)>
|
|
|
|
A promise that resolves to an object
|
|
containing the new version number of the table
|
|
|
|
***
|
|
|
|
### addColumns()
|
|
|
|
```ts
|
|
abstract addColumns(newColumnTransforms): Promise<AddColumnsResult>
|
|
```
|
|
|
|
Add new columns with defined values.
|
|
|
|
The `{ computed }` form stores the expression rather than evaluating it
|
|
now: the column is committed with no values, and rows get them from
|
|
[Table#refreshColumn](Table.md#refreshcolumn). Declaring one therefore costs the same on a
|
|
large table as on an empty one.
|
|
|
|
A refresh does not revisit rows it has already filled, so mutating an
|
|
input leaves the value computed at fill time; recomputing means dropping
|
|
the column and declaring it again. While a declaration reads a column,
|
|
that column cannot be renamed, retyped or dropped.
|
|
|
|
On LanceDB Cloud and Enterprise the expression is planned by the
|
|
server, and the refresh runs as a server job -- see
|
|
[Table#refreshColumnAsync](Table.md#refreshcolumnasync).
|
|
|
|
#### Parameters
|
|
|
|
* **newColumnTransforms**:
|
|
\| `Field`<`any`>
|
|
\| `Field`<`any`>[]
|
|
\| `Schema`<`any`>
|
|
\| [`AddColumnsSql`](../interfaces/AddColumnsSql.md)[]
|
|
\| `object`
|
|
Either:
|
|
- An array of objects with column names and SQL expressions to calculate values
|
|
- A single Arrow Field defining one column with its data type (column will be initialized with null values)
|
|
- An array of Arrow Fields defining columns with their data types (columns will be initialized with null values)
|
|
- An Arrow Schema defining columns with their data types (columns will be initialized with null values)
|
|
- `{ computed }`, declaring columns defined by a SQL expression whose type and inputs are derived from it
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`AddColumnsResult`](../interfaces/AddColumnsResult.md)>
|
|
|
|
A promise that resolves to an object
|
|
containing the new version number of the table after adding the columns.
|
|
|
|
#### Example
|
|
|
|
```ts
|
|
await table.addColumns({ computed: [{ name: "doubled", valueSql: "x * 2" }] });
|
|
const { rowsFilled } = await table.refreshColumn("doubled");
|
|
```
|
|
|
|
***
|
|
|
|
### alterColumns()
|
|
|
|
```ts
|
|
abstract alterColumns(columnAlterations): Promise<AlterColumnsResult>
|
|
```
|
|
|
|
Alter the name or nullability of columns.
|
|
|
|
#### Parameters
|
|
|
|
* **columnAlterations**: [`ColumnAlteration`](../interfaces/ColumnAlteration.md)[]
|
|
One or more alterations to
|
|
apply to columns.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`AlterColumnsResult`](../interfaces/AlterColumnsResult.md)>
|
|
|
|
A promise that resolves to an object
|
|
containing the new version number of the table after altering the columns.
|
|
|
|
***
|
|
|
|
### branches()
|
|
|
|
```ts
|
|
abstract branches(): Promise<Branches>
|
|
```
|
|
|
|
Get the branch manager for this table.
|
|
|
|
Branches are isolated, writable lines of history forked from another
|
|
branch (or version). Writes on a branch do not affect `main`.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`Branches`](Branches.md)>
|
|
|
|
***
|
|
|
|
### checkout()
|
|
|
|
```ts
|
|
abstract checkout(version): Promise<void>
|
|
```
|
|
|
|
Checks out a specific version of the table _This is an in-place operation._
|
|
|
|
This allows viewing previous versions of the table. If you wish to
|
|
keep writing to the dataset starting from an old version, then use
|
|
the `restore` function.
|
|
|
|
Calling this method will set the table into time-travel mode. If you
|
|
wish to return to standard mode, call `checkoutLatest`.
|
|
|
|
#### Parameters
|
|
|
|
* **version**: `string` \| `number`
|
|
The version to checkout, could be version number or tag
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
#### Example
|
|
|
|
```typescript
|
|
import * as lancedb from "@lancedb/lancedb"
|
|
const db = await lancedb.connect("./.lancedb");
|
|
const table = await db.createTable("my_table", [
|
|
{ vector: [1.1, 0.9], type: "vector" },
|
|
]);
|
|
|
|
console.log(await table.version()); // 1
|
|
console.log(table.display());
|
|
await table.add([{ vector: [0.5, 0.2], type: "vector" }]);
|
|
await table.checkout(1);
|
|
console.log(await table.version()); // 2
|
|
```
|
|
|
|
***
|
|
|
|
### checkoutLatest()
|
|
|
|
```ts
|
|
abstract checkoutLatest(): Promise<void>
|
|
```
|
|
|
|
Checkout the latest version of the table. _This is an in-place operation._
|
|
|
|
The table will be set back into standard mode, and will track the latest
|
|
version of the table.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
***
|
|
|
|
### checkpointLsm()
|
|
|
|
```ts
|
|
abstract checkpointLsm(): Promise<void>
|
|
```
|
|
|
|
Converge this table's LSM write path into its base table.
|
|
|
|
Seals once, then triggers compaction and polls until the L0 that existed
|
|
at the start is gone. The target set is fixed at the start, so
|
|
generations created *during* the checkpoint are ignored — that is what
|
|
lets it terminate under write load, and what makes it best-effort: it
|
|
converges the fresh tier as of some instant. Idempotent, abandonable at
|
|
any point, and safe to run on a cadence.
|
|
|
|
There is no liveness bound — the compactor pool is shared across tables,
|
|
so a checkpoint queued behind unrelated work looks exactly like one that
|
|
is merging. The caller owns the deadline.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
#### Example
|
|
|
|
```ts
|
|
const before = await table.getLsmStats();
|
|
await table.checkpointLsm();
|
|
const after = await table.getLsmStats();
|
|
```
|
|
|
|
***
|
|
|
|
### close()
|
|
|
|
```ts
|
|
abstract close(): void
|
|
```
|
|
|
|
Close the table, releasing any underlying resources.
|
|
|
|
It is safe to call this method multiple times.
|
|
|
|
Any attempt to use the table after it is closed will result in an error.
|
|
|
|
#### Returns
|
|
|
|
`void`
|
|
|
|
***
|
|
|
|
### closeLsmWriters()
|
|
|
|
```ts
|
|
abstract closeLsmWriters(): Promise<void>
|
|
```
|
|
|
|
Drain and close any cached MemWAL shard writers held for this table.
|
|
|
|
When an [LsmWriteSpec](../interfaces/LsmWriteSpec.md) is installed, `mergeInsert` opens MemWAL
|
|
shard writers and caches them for reuse across calls. This closes them,
|
|
flushing pending data; writers reopen lazily on the next `mergeInsert`.
|
|
It is a no-op when no writers are cached.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
***
|
|
|
|
### compactLsm()
|
|
|
|
```ts
|
|
abstract compactLsm(): Promise<void>
|
|
```
|
|
|
|
Trigger a background L0 → base compaction pass per bucket.
|
|
|
|
Returns once the passes are *dispatched*, not once they finish — watch
|
|
[Table#getLsmStats](Table.md#getlsmstats) for progress, or use
|
|
[Table#checkpointLsm](Table.md#checkpointlsm) to wait for convergence.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
***
|
|
|
|
### countRows()
|
|
|
|
```ts
|
|
abstract countRows(filter?): Promise<number>
|
|
```
|
|
|
|
Count the total number of rows in the dataset.
|
|
|
|
#### Parameters
|
|
|
|
* **filter?**: `string`
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`number`>
|
|
|
|
***
|
|
|
|
### createIndex()
|
|
|
|
```ts
|
|
abstract createIndex(column, options?): Promise<void>
|
|
```
|
|
|
|
Create an index to speed up queries.
|
|
|
|
Indices can be created on vector columns or scalar columns.
|
|
Indices on vector columns will speed up vector searches.
|
|
Indices on scalar columns will speed up filtering (in both
|
|
vector and non-vector searches)
|
|
|
|
We currently don't support custom named indexes.
|
|
The index name will always be `${column}_idx`.
|
|
|
|
#### Parameters
|
|
|
|
* **column**: `string`
|
|
|
|
* **options?**: `Partial`<[`IndexOptions`](../interfaces/IndexOptions.md)>
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
#### Examples
|
|
|
|
```ts
|
|
// If the column has a vector (fixed size list) data type then
|
|
// an IvfPq vector index will be created.
|
|
const table = await conn.openTable("my_table");
|
|
await table.createIndex("vector");
|
|
```
|
|
|
|
```ts
|
|
// For advanced control over vector index creation you can specify
|
|
// the index type and options.
|
|
const table = await conn.openTable("my_table");
|
|
await table.createIndex("vector", {
|
|
config: lancedb.Index.ivfPq({
|
|
numPartitions: 128,
|
|
numSubVectors: 16,
|
|
}),
|
|
});
|
|
```
|
|
|
|
```ts
|
|
// Or create a Scalar index
|
|
await table.createIndex("my_float_col");
|
|
```
|
|
|
|
***
|
|
|
|
### createIndexAsync()
|
|
|
|
```ts
|
|
abstract createIndexAsync(column, options?): Promise<Job>
|
|
```
|
|
|
|
Create an index, returning a handle to the indexing job.
|
|
|
|
The job may already be complete when returned; callers must not assume
|
|
the index exists until [Job.wait](Job.md#wait) resolves.
|
|
|
|
#### Parameters
|
|
|
|
* **column**: `string`
|
|
|
|
* **options?**: `Partial`<[`IndexOptions`](../interfaces/IndexOptions.md)>
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`Job`](Job.md)>
|
|
|
|
***
|
|
|
|
### currentBranch()
|
|
|
|
```ts
|
|
abstract currentBranch(): null | string
|
|
```
|
|
|
|
The branch this table handle is scoped to, or `null` for the main branch.
|
|
|
|
A handle returned by [Branches.create](Branches.md#create) or [Branches.checkout](Branches.md#checkout)
|
|
reports the branch it targets; a handle opened normally reports `null`.
|
|
|
|
#### Returns
|
|
|
|
`null` \| `string`
|
|
|
|
***
|
|
|
|
### delete()
|
|
|
|
```ts
|
|
abstract delete(predicate): Promise<DeleteResult>
|
|
```
|
|
|
|
Delete the rows that satisfy the predicate.
|
|
|
|
#### Parameters
|
|
|
|
* **predicate**: `string`
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`DeleteResult`](../interfaces/DeleteResult.md)>
|
|
|
|
A promise that resolves to an object
|
|
containing the new version number of the table
|
|
|
|
***
|
|
|
|
### display()
|
|
|
|
```ts
|
|
abstract display(): string
|
|
```
|
|
|
|
Return a brief description of the table
|
|
|
|
#### Returns
|
|
|
|
`string`
|
|
|
|
***
|
|
|
|
### dropColumns()
|
|
|
|
```ts
|
|
abstract dropColumns(columnNames): Promise<DropColumnsResult>
|
|
```
|
|
|
|
Drop one or more columns from the dataset
|
|
|
|
This is a metadata-only operation and does not remove the data from the
|
|
underlying storage. In order to remove the data, you must subsequently
|
|
call ``compact_files`` to rewrite the data without the removed columns and
|
|
then call ``cleanup_files`` to remove the old files.
|
|
|
|
#### Parameters
|
|
|
|
* **columnNames**: `string`[]
|
|
The names of the columns to drop. These can
|
|
be nested column references (e.g. "a.b.c") or top-level column names
|
|
(e.g. "a").
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`DropColumnsResult`](../interfaces/DropColumnsResult.md)>
|
|
|
|
A promise that resolves to an object
|
|
containing the new version number of the table after dropping the columns.
|
|
|
|
***
|
|
|
|
### dropIndex()
|
|
|
|
```ts
|
|
abstract dropIndex(name): Promise<void>
|
|
```
|
|
|
|
Drop an index from the table.
|
|
|
|
#### Parameters
|
|
|
|
* **name**: `string`
|
|
The name of the index.
|
|
This does not delete the index from disk, it just removes it from the table.
|
|
To delete the index, run [Table#optimize](Table.md#optimize) after dropping the index.
|
|
Use [Table.listIndices](Table.md#listindices) to find the names of the indices.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
***
|
|
|
|
### flushLsm()
|
|
|
|
```ts
|
|
abstract flushLsm(): Promise<void>
|
|
```
|
|
|
|
Seal every bucket's active memtable into a new L0 generation.
|
|
|
|
Returns once the seal is committed. Sealing an empty memtable is a no-op,
|
|
so this is safe to call repeatedly.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
***
|
|
|
|
### getLsmStats()
|
|
|
|
```ts
|
|
abstract getLsmStats(includeGenerationRows?): Promise<undefined | LsmStats>
|
|
```
|
|
|
|
Read live per-bucket LSM state.
|
|
|
|
Answers "how far behind is my fresh tier", "which bucket is hot", and
|
|
"why is my fresh-tier vector search brute-force". Mutates no table state.
|
|
|
|
Resolves to `undefined` only when the LSM write path is not enabled.
|
|
|
|
#### Parameters
|
|
|
|
* **includeGenerationRows?**: `boolean`
|
|
Also count rows per L0 generation.
|
|
Off by default because each count opens an uncached Lance dataset.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`undefined` \| [`LsmStats`](../interfaces/LsmStats.md)>
|
|
|
|
***
|
|
|
|
### getLsmWriteSpec()
|
|
|
|
```ts
|
|
abstract getLsmWriteSpec(): Promise<undefined | LsmWriteSpec>
|
|
```
|
|
|
|
Read the [LsmWriteSpec](../interfaces/LsmWriteSpec.md) currently installed on this table.
|
|
|
|
Resolves to `undefined` when the MemWAL LSM write path is not enabled (no
|
|
spec has been set, or it was removed with [Table#unsetLsmWriteSpec](Table.md#unsetlsmwritespec)).
|
|
The returned spec mirrors what was passed to
|
|
[Table#setLsmWriteSpec](Table.md#setlsmwritespec), except that `maintainedIndexes` always
|
|
reports the concrete list resolved when the spec was set — `undefined`
|
|
never round-trips.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`undefined` \| [`LsmWriteSpec`](../interfaces/LsmWriteSpec.md)>
|
|
|
|
***
|
|
|
|
### indexStats()
|
|
|
|
```ts
|
|
abstract indexStats(name): Promise<undefined | IndexStatistics>
|
|
```
|
|
|
|
List all the stats of a specified index
|
|
|
|
#### Parameters
|
|
|
|
* **name**: `string`
|
|
The name of the index.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`undefined` \| [`IndexStatistics`](../interfaces/IndexStatistics.md)>
|
|
|
|
The stats of the index. If the index does not exist, it will return undefined
|
|
|
|
Use [Table.listIndices](Table.md#listindices) to find the names of the indices.
|
|
|
|
***
|
|
|
|
### initialStorageOptions()
|
|
|
|
```ts
|
|
abstract initialStorageOptions(): Promise<undefined | null | Record<string, string>>
|
|
```
|
|
|
|
Get the initial storage options that were passed in when opening this table.
|
|
|
|
For dynamically refreshed options (e.g., credential vending), use
|
|
[Table.latestStorageOptions](Table.md#lateststorageoptions).
|
|
|
|
Warning: This is an internal API and the return value is subject to change.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`undefined` \| `null` \| `Record`<`string`, `string`>>
|
|
|
|
The storage options, or undefined if no storage options were configured.
|
|
|
|
***
|
|
|
|
### isOpen()
|
|
|
|
```ts
|
|
abstract isOpen(): boolean
|
|
```
|
|
|
|
Return true if the table has not been closed
|
|
|
|
#### Returns
|
|
|
|
`boolean`
|
|
|
|
***
|
|
|
|
### latestStorageOptions()
|
|
|
|
```ts
|
|
abstract latestStorageOptions(): Promise<undefined | null | Record<string, string>>
|
|
```
|
|
|
|
Get the latest storage options, refreshing from provider if configured.
|
|
|
|
This method is useful for credential vending scenarios where storage options
|
|
may be refreshed dynamically. If no dynamic provider is configured, this
|
|
returns the initial static options.
|
|
|
|
Warning: This is an internal API and the return value is subject to change.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`undefined` \| `null` \| `Record`<`string`, `string`>>
|
|
|
|
The storage options, or undefined if no storage options were configured.
|
|
|
|
***
|
|
|
|
### listIndices()
|
|
|
|
```ts
|
|
abstract listIndices(): Promise<IndexConfig[]>
|
|
```
|
|
|
|
List all indices that have been created with [Table.createIndex](Table.md#createindex)
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`IndexConfig`](../interfaces/IndexConfig.md)[]>
|
|
|
|
***
|
|
|
|
### listVersions()
|
|
|
|
```ts
|
|
abstract listVersions(): Promise<Version[]>
|
|
```
|
|
|
|
List all the versions of the table
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`Version`](../interfaces/Version.md)[]>
|
|
|
|
***
|
|
|
|
### mergeInsert()
|
|
|
|
```ts
|
|
abstract mergeInsert(on): MergeInsertBuilder
|
|
```
|
|
|
|
#### Parameters
|
|
|
|
* **on**: `string` \| `string`[]
|
|
|
|
#### Returns
|
|
|
|
[`MergeInsertBuilder`](MergeInsertBuilder.md)
|
|
|
|
***
|
|
|
|
### optimize()
|
|
|
|
```ts
|
|
abstract optimize(options?): Promise<OptimizeStats>
|
|
```
|
|
|
|
Optimize the on-disk data and indices for better performance.
|
|
|
|
Modeled after ``VACUUM`` in PostgreSQL.
|
|
|
|
Optimization covers three operations:
|
|
|
|
- Compaction: Merges small files into larger ones
|
|
- Prune: Removes old versions of the dataset
|
|
- Index: Optimizes the indices, adding new data to existing indices
|
|
|
|
The frequency an application should call optimize is based on the frequency of
|
|
data modifications. If data is frequently added, deleted, or updated then
|
|
optimize should be run frequently. A good rule of thumb is to run optimize if
|
|
you have added or modified 100,000 or more records or run more than 20 data
|
|
modification operations.
|
|
|
|
#### Parameters
|
|
|
|
* **options?**: `Partial`<[`OptimizeOptions`](../interfaces/OptimizeOptions.md)>
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`OptimizeStats`](../interfaces/OptimizeStats.md)>
|
|
|
|
***
|
|
|
|
### prewarmData()
|
|
|
|
```ts
|
|
abstract prewarmData(columns?): Promise<void>
|
|
```
|
|
|
|
Prewarm one or more columns of data in the table.
|
|
|
|
#### Parameters
|
|
|
|
* **columns?**: `string`[]
|
|
The columns to prewarm. If undefined, all columns are prewarmed.
|
|
This will load the column data into the page cache so that future queries that
|
|
read those columns avoid the initial cold-start latency. This call initiates
|
|
prewarming and returns once the request is accepted; the warming itself may
|
|
continue in the background. Calling it on already-prewarmed columns is a
|
|
no-op on the server.
|
|
Prewarming is generally useful for columns used in filters or projections.
|
|
Large columns (e.g. high-dimensional vectors or binary data) may not be
|
|
practical to prewarm.
|
|
This feature is currently only supported on remote tables.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
***
|
|
|
|
### prewarmIndex()
|
|
|
|
```ts
|
|
abstract prewarmIndex(name): Promise<void>
|
|
```
|
|
|
|
Prewarm an index in the table.
|
|
|
|
#### Parameters
|
|
|
|
* **name**: `string`
|
|
The name of the index.
|
|
This will load the index into memory. This may reduce the cold-start time for
|
|
future queries. If the index does not fit in the cache then this call may be
|
|
wasteful.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
***
|
|
|
|
### query()
|
|
|
|
```ts
|
|
abstract query(): Query
|
|
```
|
|
|
|
Create a [Query](Query.md) Builder.
|
|
|
|
Queries allow you to search your existing data. By default the query will
|
|
return all the data in the table in no particular order. The builder
|
|
returned by this method can be used to control the query using filtering,
|
|
vector similarity, sorting, and more.
|
|
|
|
Note: By default, all columns are returned. For best performance, you should
|
|
only fetch the columns you need.
|
|
|
|
When appropriate, various indices and statistics based pruning will be used to
|
|
accelerate the query.
|
|
|
|
#### Returns
|
|
|
|
[`Query`](Query.md)
|
|
|
|
A builder that can be used to parameterize the query
|
|
|
|
#### Examples
|
|
|
|
```ts
|
|
// SQL-style filtering
|
|
//
|
|
// This query will return up to 1000 rows whose value in the `id` column
|
|
// is greater than 5. LanceDb supports a broad set of filtering functions.
|
|
for await (const batch of table
|
|
.query()
|
|
.where("id > 1")
|
|
.select(["id"])
|
|
.limit(20)) {
|
|
console.log(batch);
|
|
}
|
|
```
|
|
|
|
```ts
|
|
// Vector Similarity Search
|
|
//
|
|
// This example will find the 10 rows whose value in the "vector" column are
|
|
// closest to the query vector [1.0, 2.0, 3.0]. If an index has been created
|
|
// on the "vector" column then this will perform an ANN search.
|
|
//
|
|
// The `refineFactor` and `nprobes` methods are used to control the recall /
|
|
// latency tradeoff of the search.
|
|
for await (const batch of table
|
|
.query()
|
|
.where("id > 1")
|
|
.select(["id"])
|
|
.limit(20)) {
|
|
console.log(batch);
|
|
}
|
|
```
|
|
|
|
```ts
|
|
// Scan the full dataset
|
|
//
|
|
// This query will return everything in the table in no particular order.
|
|
for await (const batch of table.query()) {
|
|
console.log(batch);
|
|
}
|
|
```
|
|
|
|
***
|
|
|
|
### refreshColumn()
|
|
|
|
```ts
|
|
abstract refreshColumn(column): Promise<RefreshColumnResult>
|
|
```
|
|
|
|
Fill the rows of a computed column that hold no value yet.
|
|
|
|
Rows appended since the last refresh are filled by the next one; rows
|
|
already filled are left as they are, so the call is idempotent and does
|
|
not observe a mutated input. Local tables only: a remote refresh runs
|
|
as a server job, through [Table#refreshColumnAsync](Table.md#refreshcolumnasync).
|
|
|
|
#### Parameters
|
|
|
|
* **column**: `string`
|
|
The name of the computed column to fill.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`RefreshColumnResult`](../interfaces/RefreshColumnResult.md)>
|
|
|
|
A promise that resolves to the
|
|
number of rows filled and the new version number of the table.
|
|
|
|
***
|
|
|
|
### refreshColumnAsync()
|
|
|
|
```ts
|
|
abstract refreshColumnAsync(column): Promise<Job>
|
|
```
|
|
|
|
Like [Table#refreshColumn](Table.md#refreshcolumn), but returns a handle to the refresh
|
|
job instead of blocking until it completes.
|
|
|
|
The job may already be complete when returned; callers must not assume
|
|
the column is filled until [Job.wait](Job.md#wait) resolves. Invalid input --
|
|
an unknown column, or one that is not computed -- rejects here rather
|
|
than failing the job. On local tables the job runs in-process; on
|
|
LanceDB Cloud and Enterprise it is the server's backfill job.
|
|
|
|
#### Parameters
|
|
|
|
* **column**: `string`
|
|
The name of the computed column to fill.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`Job`](Job.md)>
|
|
|
|
#### Example
|
|
|
|
```ts
|
|
const job = await table.refreshColumnAsync("doubled");
|
|
await job.wait();
|
|
console.log(await job.status()); // "finished"
|
|
```
|
|
|
|
***
|
|
|
|
### restore()
|
|
|
|
```ts
|
|
abstract restore(): Promise<void>
|
|
```
|
|
|
|
Restore the table to the currently checked out version
|
|
|
|
This operation will fail if checkout has not been called previously
|
|
|
|
This operation will overwrite the latest version of the table with a
|
|
previous version. Any changes made since the checked out version will
|
|
no longer be visible.
|
|
|
|
Once the operation concludes the table will no longer be in a checked
|
|
out state and the read_consistency_interval, if any, will apply.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
***
|
|
|
|
### schema()
|
|
|
|
```ts
|
|
abstract schema(): Promise<Schema<any>>
|
|
```
|
|
|
|
Get the schema of the table.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`Schema`<`any`>>
|
|
|
|
***
|
|
|
|
### search()
|
|
|
|
```ts
|
|
abstract search(
|
|
query,
|
|
queryType?,
|
|
ftsColumns?): Query | VectorQuery | AutoQuery
|
|
```
|
|
|
|
Create a search query to find the nearest neighbors
|
|
of the given query
|
|
|
|
#### Parameters
|
|
|
|
* **query**: `string` \| [`IntoVector`](../type-aliases/IntoVector.md) \| [`MultiVector`](../type-aliases/MultiVector.md) \| [`FullTextQuery`](../interfaces/FullTextQuery.md)
|
|
the query, a vector or string
|
|
|
|
* **queryType?**: `string`
|
|
the type of the query, "vector", "fts", or "auto"
|
|
|
|
* **ftsColumns?**: `string` \| `string`[]
|
|
the columns to search in for full text search
|
|
for now, only one column can be searched at a time.
|
|
when "auto" is used, if the query is a string and an embedding function is defined, it will be treated as a vector query
|
|
if the query is a string and no embedding function is defined, it will be treated as a full text search query
|
|
|
|
#### Returns
|
|
|
|
[`Query`](Query.md) \| [`VectorQuery`](VectorQuery.md) \| [`AutoQuery`](AutoQuery.md)
|
|
|
|
***
|
|
|
|
### setLsmWriteSpec()
|
|
|
|
```ts
|
|
abstract setLsmWriteSpec(spec): Promise<void>
|
|
```
|
|
|
|
Install an [LsmWriteSpec](../interfaces/LsmWriteSpec.md) on this table, selecting Lance's MemWAL
|
|
LSM-style write path for future `mergeInsert` calls.
|
|
|
|
`LsmWriteSpec` chooses one of three sharding strategies via `specType`:
|
|
|
|
- `"bucket"` — hash-bucket writes by the single-column unenforced primary
|
|
key (`column` and `numBuckets` required).
|
|
- `"identity"` — shard by the raw value of a scalar `column`.
|
|
- `"unsharded"` — route every write to a single shard.
|
|
|
|
All variants require the table to have an unenforced primary key
|
|
([Table#setUnenforcedPrimaryKey](Table.md#setunenforcedprimarykey)); bucket sharding additionally
|
|
requires it to be the single column being bucketed.
|
|
|
|
Omitting `maintainedIndexes` maintains every index on the table, resolved
|
|
here, failing if one cannot be maintained — name them to install anyway.
|
|
Naming them pins an exact set, and a still-building index is rejected
|
|
rather than quietly omitted.
|
|
|
|
#### Parameters
|
|
|
|
* **spec**: [`LsmWriteSpec`](../interfaces/LsmWriteSpec.md)
|
|
The sharding spec to install.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
#### Example
|
|
|
|
```ts
|
|
await table.setUnenforcedPrimaryKey("id");
|
|
await table.setLsmWriteSpec({
|
|
specType: "bucket",
|
|
column: "id",
|
|
numBuckets: 16,
|
|
maintainedIndexes: ["id_idx"],
|
|
});
|
|
```
|
|
|
|
***
|
|
|
|
### setUnenforcedPrimaryKey()
|
|
|
|
```ts
|
|
abstract setUnenforcedPrimaryKey(columns): Promise<void>
|
|
```
|
|
|
|
Set the unenforced primary key for this table to a single column.
|
|
|
|
"Unenforced" means LanceDB does not check uniqueness on writes; the
|
|
column is recorded in the schema as the primary key for use by features
|
|
such as `merge_insert`. Only single-column primary keys are supported,
|
|
and the key cannot be changed once set.
|
|
|
|
#### Parameters
|
|
|
|
* **columns**: `string` \| `string`[]
|
|
The primary key column. A one-element
|
|
array is also accepted; passing more than one column is rejected.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
***
|
|
|
|
### stats()
|
|
|
|
```ts
|
|
abstract stats(): Promise<TableStatistics>
|
|
```
|
|
|
|
Returns table and fragment statistics
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`TableStatistics`](../interfaces/TableStatistics.md)>
|
|
|
|
The table and fragment statistics
|
|
|
|
***
|
|
|
|
### tags()
|
|
|
|
```ts
|
|
abstract tags(): Promise<Tags>
|
|
```
|
|
|
|
Get a tags manager for this table.
|
|
|
|
Tags allow you to label specific versions of a table with a human-readable name.
|
|
The returned tags manager can be used to list, create, update, or delete tags.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`Tags`](Tags.md)>
|
|
|
|
A tags manager for this table
|
|
|
|
#### Example
|
|
|
|
```typescript
|
|
const tagsManager = await table.tags();
|
|
await tagsManager.create("v1", 1);
|
|
const tags = await tagsManager.list();
|
|
console.log(tags); // { "v1": { version: 1, manifestSize: ... } }
|
|
```
|
|
|
|
***
|
|
|
|
### takeOffsets()
|
|
|
|
```ts
|
|
abstract takeOffsets(offsets): TakeQuery
|
|
```
|
|
|
|
Create a query that returns a subset of the rows in the table.
|
|
|
|
#### Parameters
|
|
|
|
* **offsets**: `number`[]
|
|
The offsets of the rows to return.
|
|
|
|
#### Returns
|
|
|
|
[`TakeQuery`](TakeQuery.md)
|
|
|
|
A builder that can be used to parameterize the query.
|
|
|
|
***
|
|
|
|
### takeRowIds()
|
|
|
|
```ts
|
|
abstract takeRowIds(rowIds): TakeQuery
|
|
```
|
|
|
|
Create a query that returns a subset of the rows in the table.
|
|
|
|
#### Parameters
|
|
|
|
* **rowIds**: readonly (`number` \| `bigint`)[]
|
|
The row ids of the rows to return.
|
|
Row ids returned by `withRowId()` are `bigint`, so `bigint[]` is supported.
|
|
For convenience / backwards compatibility, `number[]` is also accepted (for
|
|
small row ids that fit in a safe integer).
|
|
|
|
#### Returns
|
|
|
|
[`TakeQuery`](TakeQuery.md)
|
|
|
|
A builder that can be used to parameterize the query.
|
|
|
|
***
|
|
|
|
### toArrow()
|
|
|
|
```ts
|
|
abstract toArrow(): Promise<Table<any>>
|
|
```
|
|
|
|
Return the table as an arrow table
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`Table`<`any`>>
|
|
|
|
***
|
|
|
|
### tokenize()
|
|
|
|
```ts
|
|
abstract tokenize(query, options): Promise<FtsToken[]>
|
|
```
|
|
|
|
Tokenize a full-text search query using the tokenizer configured on an FTS index.
|
|
|
|
Specify exactly one of `column` or `indexName`.
|
|
|
|
Model-backed tokenizers such as `jieba/*` and `lindera/*` are rebuilt in
|
|
the client process from index metadata. For remote tables, this means the
|
|
same tokenizer model files must also exist locally.
|
|
|
|
#### Parameters
|
|
|
|
* **query**: `string`
|
|
|
|
* **options**: [`TokenizeTableOptions`](../type-aliases/TokenizeTableOptions.md)
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`FtsToken`](../interfaces/FtsToken.md)[]>
|
|
|
|
***
|
|
|
|
### unsetLsmWriteSpec()
|
|
|
|
```ts
|
|
abstract unsetLsmWriteSpec(): Promise<void>
|
|
```
|
|
|
|
Remove the [LsmWriteSpec](../interfaces/LsmWriteSpec.md) from this table, reverting to the standard
|
|
`mergeInsert` write path.
|
|
|
|
Errors if no spec is currently set.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|
|
|
|
***
|
|
|
|
### update()
|
|
|
|
#### update(opts)
|
|
|
|
```ts
|
|
abstract update(opts): Promise<UpdateResult>
|
|
```
|
|
|
|
Update existing records in the Table
|
|
|
|
##### Parameters
|
|
|
|
* **opts**: `object` & `Partial`<[`UpdateOptions`](../interfaces/UpdateOptions.md)>
|
|
|
|
##### Returns
|
|
|
|
`Promise`<[`UpdateResult`](../interfaces/UpdateResult.md)>
|
|
|
|
A promise that resolves to an object containing
|
|
the number of rows updated and the new version number
|
|
|
|
##### Example
|
|
|
|
```ts
|
|
table.update({where:"x = 2", values:{"vector": [10, 10]}})
|
|
```
|
|
|
|
#### update(opts)
|
|
|
|
```ts
|
|
abstract update(opts): Promise<UpdateResult>
|
|
```
|
|
|
|
Update existing records in the Table
|
|
|
|
##### Parameters
|
|
|
|
* **opts**: `object` & `Partial`<[`UpdateOptions`](../interfaces/UpdateOptions.md)>
|
|
|
|
##### Returns
|
|
|
|
`Promise`<[`UpdateResult`](../interfaces/UpdateResult.md)>
|
|
|
|
A promise that resolves to an object containing
|
|
the number of rows updated and the new version number
|
|
|
|
##### Example
|
|
|
|
```ts
|
|
table.update({where:"x = 2", valuesSql:{"x": "x + 1"}})
|
|
```
|
|
|
|
#### update(updates, options)
|
|
|
|
```ts
|
|
abstract update(updates, options?): Promise<UpdateResult>
|
|
```
|
|
|
|
Update existing records in the Table
|
|
|
|
An update operation can be used to adjust existing values. Use the
|
|
returned builder to specify which columns to update. The new value
|
|
can be a literal value (e.g. replacing nulls with some default value)
|
|
or an expression applied to the old value (e.g. incrementing a value)
|
|
|
|
An optional condition can be specified (e.g. "only update if the old
|
|
value is 0")
|
|
|
|
Note: if your condition is something like "some_id_column == 7" and
|
|
you are updating many rows (with different ids) then you will get
|
|
better performance with a single [`merge_insert`] call instead of
|
|
repeatedly calilng this method.
|
|
|
|
##### Parameters
|
|
|
|
* **updates**: `Record`<`string`, `string`> \| `Map`<`string`, `string`>
|
|
the
|
|
columns to update
|
|
|
|
* **options?**: `Partial`<[`UpdateOptions`](../interfaces/UpdateOptions.md)>
|
|
additional options to control
|
|
the update behavior
|
|
|
|
##### Returns
|
|
|
|
`Promise`<[`UpdateResult`](../interfaces/UpdateResult.md)>
|
|
|
|
A promise that resolves to an object
|
|
containing the number of rows updated and the new version number
|
|
|
|
Keys in the map should specify the name of the column to update.
|
|
Values in the map provide the new value of the column. These can
|
|
be SQL literal strings (e.g. "7" or "'foo'") or they can be expressions
|
|
based on the row being updated (e.g. "my_col + 1")
|
|
|
|
***
|
|
|
|
### updateFieldMetadata()
|
|
|
|
```ts
|
|
abstract updateFieldMetadata(updates): Promise<UpdateFieldMetadataResult>
|
|
```
|
|
|
|
Update per-field (column) metadata.
|
|
|
|
The following keys are treated specially, by convention, and should be
|
|
used when appropriate:
|
|
|
|
- `lancedb:description`: for a human-readable description of a field.
|
|
- `lancedb:tag:<name>`: for a user-defined key-value tag, where the suffix
|
|
names the tag category; e.g. `lancedb:tag:model: "clip"`.
|
|
- `lancedb:logical-column`: for a column grouping; e.g. `feature_v1` and
|
|
`feature_v2` might be in the same logical column.
|
|
- `lancedb:status`: for status options (`production`, `candidate`,
|
|
`deprecated`, `archived`) to designate the current life cycle state of
|
|
this column.
|
|
|
|
#### Parameters
|
|
|
|
* **updates**: [`FieldMetadataUpdate`](../interfaces/FieldMetadataUpdate.md)[]
|
|
One or more per-field updates. Each
|
|
update's metadata is merged into the field's existing metadata by default;
|
|
a value of `null` deletes that key, and `replace: true` swaps the whole map.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<[`UpdateFieldMetadataResult`](../interfaces/UpdateFieldMetadataResult.md)>
|
|
|
|
resolves to the new table version.
|
|
|
|
***
|
|
|
|
### vectorSearch()
|
|
|
|
```ts
|
|
abstract vectorSearch(vector): VectorQuery
|
|
```
|
|
|
|
Search the table with a given query vector.
|
|
|
|
This is a convenience method for preparing a vector query and
|
|
is the same thing as calling `nearestTo` on the builder returned
|
|
by `query`.
|
|
|
|
#### Parameters
|
|
|
|
* **vector**: [`IntoVector`](../type-aliases/IntoVector.md) \| [`MultiVector`](../type-aliases/MultiVector.md)
|
|
|
|
#### Returns
|
|
|
|
[`VectorQuery`](VectorQuery.md)
|
|
|
|
#### See
|
|
|
|
[Query#nearestTo](Query.md#nearestto) for more details.
|
|
|
|
***
|
|
|
|
### version()
|
|
|
|
```ts
|
|
abstract version(): Promise<number>
|
|
```
|
|
|
|
Retrieve the version of the table
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`number`>
|
|
|
|
***
|
|
|
|
### waitForIndex()
|
|
|
|
```ts
|
|
abstract waitForIndex(indexNames, timeoutSeconds): Promise<void>
|
|
```
|
|
|
|
Waits for asynchronous indexing to complete on the table.
|
|
|
|
#### Parameters
|
|
|
|
* **indexNames**: `string`[]
|
|
The name of the indices to wait for
|
|
|
|
* **timeoutSeconds**: `number`
|
|
The number of seconds to wait before timing out
|
|
This will raise an error if the indices are not created and fully indexed within the timeout.
|
|
|
|
#### Returns
|
|
|
|
`Promise`<`void`>
|