[python] Bump version: 0.1.12 → 0.1.13

Bump version: 0.1.14 → 0.1.15
[Python] Support pydantic v1 as well (#337 )
2025-12-23 05:19:58 +00:00 · 2023-07-19 03:09:50 +00:00 · 2023-07-19 02:54:10 +00:00 · 2023-07-18 19:53:09 -07:00 · 2023-07-18 21:21:39 -04:00 · 2023-07-19 01:18:28 +00:00
41 changed files with 1268 additions and 449 deletions
--- a/.bumpversion.cfg
+++ b/.bumpversion.cfg
@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.1.13
+current_version = 0.1.15
 commit = True
 message = Bump version: {current_version} → {new_version}
 tag = True
--- a/.github/workflows/make-release-commit.yml
+++ b/.github/workflows/make-release-commit.yml
@@ -45,6 +45,13 @@ jobs:
      run: |
        pip install bump2version
        bumpversion --verbose ${{ inputs.part }}
+    - name: Update package-lock.json file
+      run: |
+        npm install
+        git add package-lock.json
+        # Add this change to the commit created by bumpversion
+        git commit --amend --no-edit
+      working-directory: node
    - name: Push new version and tag
      if: ${{ inputs.dry_run }} == "false"
      uses: ad-m/github-push-action@master
--- a/.github/workflows/npm-publish.yml
+++ b/.github/workflows/npm-publish.yml
@@ -161,6 +161,7 @@ jobs:
      - uses: actions/setup-node@v3
        with:
          node-version: 20
+          registry-url: 'https://registry.npmjs.org'
      - name: Publish to NPM
        env:
          NODE_AUTH_TOKEN: ${{ secrets.LANCEDB_NPM_REGISTRY_TOKEN }}
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -11,6 +11,6 @@ arrow-array = "42.0"
 arrow-data = "42.0"
 arrow-schema = "42.0"
 arrow-ipc = "42.0"
-half = { "version" = "2.2.1", default-features = false }
+half = { "version" = "=2.2.1", default-features = false }
 object_store = "0.6.1"

--- a/docs/src/basic.md
+++ b/docs/src/basic.md
@@ -122,6 +122,35 @@ After a table has been created, you can always add more data to it using
              {vector: [9.5, 56.2], item: "buzz", price: 200.0}])
      ```

+## How to delete rows from a table
+
+Use the `delete()` method on tables to delete rows from a table. To choose
+which rows to delete, provide a filter that matches on the metadata columns.
+This can delete any number of rows that match the filter.
+
+=== "Python"
+      ```python
+      tbl.delete('item = "fizz"')
+      ```
+
+=== "Javascript"
+      ```javascript
+      await tbl.delete('item = "fizz"')
+      ```
+
+The deletion predicate is a SQL expression that supports the same expressions
+as the `where()` clause on a search. They can be as simple or complex as needed.
+To see what expressions are supported, see the [SQL filters](sql.md) section.
+
+
+=== "Python"
+
+      Read more: [lancedb.table.Table.delete][]
+
+=== "Javascript"
+
+      Read more: [vectordb.Table.delete](javascript/interfaces/Table.md#delete)
+
 ## How to search for (approximate) nearest neighbors

 Once you've embedded the query, you can find its nearest neighbors using the following code:
--- a/docs/src/javascript/README.md
+++ b/docs/src/javascript/README.md
@@ -10,6 +10,10 @@ A JavaScript / Node.js library for [LanceDB](https://github.com/lancedb/lancedb)
 npm install vectordb
 ```

+This will download the appropriate native library for your platform. We currently
+support x86_64 Linux, aarch64 Linux, Intel MacOS, and ARM (M1/M2) MacOS. We do not
+yet support Windows or musl-based Linux (such as Alpine Linux).
+
 ## Usage

 ### Basic Example
@@ -28,12 +32,34 @@ The [examples](./examples) folder contains complete examples.

 ## Development

-Run the tests with
+To build everything fresh:
+
+```bash
+npm install
+npm run tsc
+npm run build
+```
+
+Then you should be able to run the tests with:

 ```bash
 npm test
 ```

+### Rebuilding Rust library
+
+```bash
+npm run build
+```
+
+### Rebuilding Typescript
+
+```bash
+npm run tsc
+```
+
+### Fix lints
+
 To run the linter and have it automatically fix all errors

 ```bash
--- a/docs/src/javascript/classes/LocalConnection.md
+++ b/docs/src/javascript/classes/LocalConnection.md
@@ -17,7 +17,7 @@ A connection to a LanceDB database.
 ### Properties

 - [\_db](LocalConnection.md#_db)
- [\_uri](LocalConnection.md#_uri)
+- [\_options](LocalConnection.md#_options)

 ### Accessors

@@ -35,18 +35,18 @@ A connection to a LanceDB database.

 ### constructor

-• **new LocalConnection**(`db`, `uri`)
+• **new LocalConnection**(`db`, `options`)

 #### Parameters

 | Name | Type |
 | :------ | :------ |
 | `db` | `any` |
-| `uri` | `string` |
+| `options` | [`ConnectionOptions`](../interfaces/ConnectionOptions.md) |

 #### Defined in

-[index.ts:132](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L132)
+[index.ts:184](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L184)

 ## Properties

@@ -56,17 +56,17 @@ A connection to a LanceDB database.

 #### Defined in

-[index.ts:130](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L130)
+[index.ts:182](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L182)

 ___

-### \_uri
+### \_options

-• `Private` `Readonly` **\_uri**: `string`
+• `Private` `Readonly` **\_options**: [`ConnectionOptions`](../interfaces/ConnectionOptions.md)

 #### Defined in

-[index.ts:129](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L129)
+[index.ts:181](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L181)

 ## Accessors

@@ -84,7 +84,7 @@ ___

 #### Defined in

-[index.ts:137](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L137)
+[index.ts:189](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L189)

 ## Methods

@@ -112,7 +112,7 @@ Creates a new Table and initialize it with new data.

 #### Defined in

-[index.ts:177](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L177)
+[index.ts:230](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L230)

 ▸ **createTable**(`name`, `data`, `mode`): `Promise`<[`Table`](../interfaces/Table.md)<`number`[]\>\>

@@ -134,7 +134,7 @@ Connection.createTable

 #### Defined in

-[index.ts:178](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L178)
+[index.ts:231](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L231)

 ▸ **createTable**<`T`\>(`name`, `data`, `mode`, `embeddings`): `Promise`<[`Table`](../interfaces/Table.md)<`T`\>\>

@@ -165,7 +165,36 @@ Connection.createTable

 #### Defined in

-[index.ts:188](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L188)
+[index.ts:241](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L241)
+
+▸ **createTable**<`T`\>(`name`, `data`, `mode`, `embeddings?`): `Promise`<[`Table`](../interfaces/Table.md)<`T`\>\>
+
+#### Type parameters
+
+| Name |
+| :------ |
+| `T` |
+
+#### Parameters
+
+| Name | Type |
+| :------ | :------ |
+| `name` | `string` |
+| `data` | `Record`<`string`, `unknown`\>[] |
+| `mode` | [`WriteMode`](../enums/WriteMode.md) |
+| `embeddings?` | [`EmbeddingFunction`](../interfaces/EmbeddingFunction.md)<`T`\> |
+
+#### Returns
+
+`Promise`<[`Table`](../interfaces/Table.md)<`T`\>\>
+
+#### Implementation of
+
+Connection.createTable
+
+#### Defined in
+
+[index.ts:242](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L242)

 ___

@@ -190,7 +219,7 @@ ___

 #### Defined in

-[index.ts:201](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L201)
+[index.ts:266](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L266)

 ___

@@ -216,7 +245,7 @@ Drop an existing table.

 #### Defined in

-[index.ts:211](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L211)
+[index.ts:276](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L276)

 ___

@@ -242,7 +271,7 @@ Open a table in the database.

 #### Defined in

-[index.ts:153](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L153)
+[index.ts:205](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L205)

 ▸ **openTable**<`T`\>(`name`, `embeddings`): `Promise`<[`Table`](../interfaces/Table.md)<`T`\>\>

@@ -271,7 +300,34 @@ Connection.openTable

 #### Defined in

-[index.ts:160](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L160)
+[index.ts:212](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L212)
+
+▸ **openTable**<`T`\>(`name`, `embeddings?`): `Promise`<[`Table`](../interfaces/Table.md)<`T`\>\>
+
+#### Type parameters
+
+| Name |
+| :------ |
+| `T` |
+
+#### Parameters
+
+| Name | Type |
+| :------ | :------ |
+| `name` | `string` |
+| `embeddings?` | [`EmbeddingFunction`](../interfaces/EmbeddingFunction.md)<`T`\> |
+
+#### Returns
+
+`Promise`<[`Table`](../interfaces/Table.md)<`T`\>\>
+
+#### Implementation of
+
+Connection.openTable
+
+#### Defined in
+
+[index.ts:213](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L213)

 ___

@@ -291,4 +347,4 @@ Get the names of all tables in the database.

 #### Defined in

-[index.ts:144](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L144)
+[index.ts:196](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L196)
--- a/docs/src/javascript/classes/LocalTable.md
+++ b/docs/src/javascript/classes/LocalTable.md
@@ -24,6 +24,7 @@ A LanceDB Table is the collection of Records. Each Record has one or more vector

 - [\_embeddings](LocalTable.md#_embeddings)
 - [\_name](LocalTable.md#_name)
+- [\_options](LocalTable.md#_options)
 - [\_tbl](LocalTable.md#_tbl)

 ### Accessors
@@ -43,7 +44,7 @@ A LanceDB Table is the collection of Records. Each Record has one or more vector

 ### constructor

-• **new LocalTable**<`T`\>(`tbl`, `name`)
+• **new LocalTable**<`T`\>(`tbl`, `name`, `options`)

 #### Type parameters

@@ -57,12 +58,13 @@ A LanceDB Table is the collection of Records. Each Record has one or more vector
 | :------ | :------ |
 | `tbl` | `any` |
 | `name` | `string` |
+| `options` | [`ConnectionOptions`](../interfaces/ConnectionOptions.md) |

 #### Defined in

-[index.ts:221](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L221)
+[index.ts:287](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L287)

-• **new LocalTable**<`T`\>(`tbl`, `name`, `embeddings`)
+• **new LocalTable**<`T`\>(`tbl`, `name`, `options`, `embeddings`)

 #### Type parameters

@@ -76,11 +78,12 @@ A LanceDB Table is the collection of Records. Each Record has one or more vector
 | :------ | :------ | :------ |
 | `tbl` | `any` |  |
 | `name` | `string` |  |
+| `options` | [`ConnectionOptions`](../interfaces/ConnectionOptions.md) |  |
 | `embeddings` | [`EmbeddingFunction`](../interfaces/EmbeddingFunction.md)<`T`\> | An embedding function to use when interacting with this table |

 #### Defined in

-[index.ts:227](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L227)
+[index.ts:294](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L294)

 ## Properties

@@ -90,7 +93,7 @@ A LanceDB Table is the collection of Records. Each Record has one or more vector

 #### Defined in

-[index.ts:219](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L219)
+[index.ts:284](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L284)

 ___

@@ -100,7 +103,17 @@ ___

 #### Defined in

-[index.ts:218](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L218)
+[index.ts:283](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L283)
+
+___
+
+### \_options
+
+• `Private` `Readonly` **\_options**: [`ConnectionOptions`](../interfaces/ConnectionOptions.md)
+
+#### Defined in
+
+[index.ts:285](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L285)

 ___

@@ -110,7 +123,7 @@ ___

 #### Defined in

-[index.ts:217](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L217)
+[index.ts:282](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L282)

 ## Accessors

@@ -128,7 +141,7 @@ ___

 #### Defined in

-[index.ts:234](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L234)
+[index.ts:302](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L302)

 ## Methods

@@ -156,7 +169,7 @@ The number of rows added to the table

 #### Defined in

-[index.ts:252](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L252)
+[index.ts:320](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L320)

 ___

@@ -176,7 +189,7 @@ Returns the number of rows in this table.

 #### Defined in

-[index.ts:278](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L278)
+[index.ts:362](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L362)

 ___

@@ -194,7 +207,7 @@ VectorIndexParams.

 | Name | Type | Description |
 | :------ | :------ | :------ |
-| `indexParams` | `IvfPQIndexConfig` | The parameters of this Index, |
+| `indexParams` | [`IvfPQIndexConfig`](../interfaces/IvfPQIndexConfig.md) | The parameters of this Index, |

 #### Returns

@@ -206,7 +219,7 @@ VectorIndexParams.

 #### Defined in

-[index.ts:271](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L271)
+[index.ts:355](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L355)

 ___

@@ -232,7 +245,7 @@ Delete rows from this table.

 #### Defined in

-[index.ts:287](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L287)
+[index.ts:371](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L371)

 ___

@@ -260,7 +273,7 @@ The number of rows added to the table

 #### Defined in

-[index.ts:262](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L262)
+[index.ts:338](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L338)

 ___

@@ -286,4 +299,4 @@ Creates a search query to find the nearest neighbors of the given search term

 #### Defined in

-[index.ts:242](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L242)
+[index.ts:310](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L310)
--- a/docs/src/javascript/classes/OpenAIEmbeddingFunction.md
+++ b/docs/src/javascript/classes/OpenAIEmbeddingFunction.md
@@ -40,7 +40,7 @@ An embedding function that automatically creates vector representation for a giv

 #### Defined in

-[embedding/openai.ts:21](https://github.com/lancedb/lancedb/blob/7247834/node/src/embedding/openai.ts#L21)
+[embedding/openai.ts:21](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/embedding/openai.ts#L21)

 ## Properties

@@ -50,7 +50,7 @@ An embedding function that automatically creates vector representation for a giv

 #### Defined in

-[embedding/openai.ts:19](https://github.com/lancedb/lancedb/blob/7247834/node/src/embedding/openai.ts#L19)
+[embedding/openai.ts:19](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/embedding/openai.ts#L19)

 ___

@@ -60,7 +60,7 @@ ___

 #### Defined in

-[embedding/openai.ts:18](https://github.com/lancedb/lancedb/blob/7247834/node/src/embedding/openai.ts#L18)
+[embedding/openai.ts:18](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/embedding/openai.ts#L18)

 ___

@@ -76,7 +76,7 @@ The name of the column that will be used as input for the Embedding Function.

 #### Defined in

-[embedding/openai.ts:50](https://github.com/lancedb/lancedb/blob/7247834/node/src/embedding/openai.ts#L50)
+[embedding/openai.ts:50](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/embedding/openai.ts#L50)

 ## Methods

@@ -102,4 +102,4 @@ Creates a vector representation for the given values.

 #### Defined in

-[embedding/openai.ts:38](https://github.com/lancedb/lancedb/blob/7247834/node/src/embedding/openai.ts#L38)
+[embedding/openai.ts:38](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/embedding/openai.ts#L38)
--- a/docs/src/javascript/classes/Query.md
+++ b/docs/src/javascript/classes/Query.md
@@ -62,7 +62,7 @@ A builder for nearest neighbor queries for LanceDB.

 #### Defined in

-[index.ts:362](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L362)
+[index.ts:448](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L448)

 ## Properties

@@ -72,7 +72,7 @@ A builder for nearest neighbor queries for LanceDB.

 #### Defined in

-[index.ts:360](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L360)
+[index.ts:446](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L446)

 ___

@@ -82,7 +82,7 @@ ___

 #### Defined in

-[index.ts:358](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L358)
+[index.ts:444](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L444)

 ___

@@ -92,7 +92,7 @@ ___

 #### Defined in

-[index.ts:354](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L354)
+[index.ts:440](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L440)

 ___

@@ -102,7 +102,7 @@ ___

 #### Defined in

-[index.ts:359](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L359)
+[index.ts:445](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L445)

 ___

@@ -112,7 +112,7 @@ ___

 #### Defined in

-[index.ts:356](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L356)
+[index.ts:442](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L442)

 ___

@@ -122,7 +122,7 @@ ___

 #### Defined in

-[index.ts:352](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L352)
+[index.ts:438](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L438)

 ___

@@ -132,7 +132,7 @@ ___

 #### Defined in

-[index.ts:353](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L353)
+[index.ts:439](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L439)

 ___

@@ -142,7 +142,7 @@ ___

 #### Defined in

-[index.ts:355](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L355)
+[index.ts:441](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L441)

 ___

@@ -152,7 +152,7 @@ ___

 #### Defined in

-[index.ts:357](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L357)
+[index.ts:443](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L443)

 ___

@@ -162,7 +162,7 @@ ___

 #### Defined in

-[index.ts:351](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L351)
+[index.ts:437](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L437)

 ___

@@ -188,7 +188,7 @@ A filter statement to be applied to this query.

 #### Defined in

-[index.ts:410](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L410)
+[index.ts:496](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L496)

 ## Methods

@@ -210,7 +210,7 @@ Execute the query and return the results as an Array of Objects

 #### Defined in

-[index.ts:433](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L433)
+[index.ts:519](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L519)

 ___

@@ -232,7 +232,7 @@ A filter statement to be applied to this query.

 #### Defined in

-[index.ts:405](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L405)
+[index.ts:491](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L491)

 ___

@@ -254,7 +254,7 @@ Sets the number of results that will be returned

 #### Defined in

-[index.ts:378](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L378)
+[index.ts:464](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L464)

 ___

@@ -280,7 +280,7 @@ MetricType for the different options

 #### Defined in

-[index.ts:425](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L425)
+[index.ts:511](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L511)

 ___

@@ -302,7 +302,7 @@ The number of probes used. A higher number makes search more accurate but also s

 #### Defined in

-[index.ts:396](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L396)
+[index.ts:482](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L482)

 ___

@@ -324,7 +324,7 @@ Refine the results by reading extra elements and re-ranking them in memory.

 #### Defined in

-[index.ts:387](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L387)
+[index.ts:473](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L473)

 ___

@@ -346,4 +346,4 @@ Return only the specified columns.

 #### Defined in

-[index.ts:416](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L416)
+[index.ts:502](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L502)
--- a/docs/src/javascript/enums/MetricType.md
+++ b/docs/src/javascript/enums/MetricType.md
@@ -22,7 +22,7 @@ Cosine distance

 #### Defined in

-[index.ts:481](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L481)
+[index.ts:567](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L567)

 ___

@@ -34,7 +34,7 @@ Dot product

 #### Defined in

-[index.ts:486](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L486)
+[index.ts:572](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L572)

 ___

@@ -46,4 +46,4 @@ Euclidean distance

 #### Defined in

-[index.ts:476](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L476)
+[index.ts:562](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L562)
--- a/docs/src/javascript/enums/WriteMode.md
+++ b/docs/src/javascript/enums/WriteMode.md
@@ -22,7 +22,7 @@ Append new data to the table.

 #### Defined in

-[index.ts:466](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L466)
+[index.ts:552](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L552)

 ___

@@ -34,7 +34,7 @@ Create a new [Table](../interfaces/Table.md).

 #### Defined in

-[index.ts:462](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L462)
+[index.ts:548](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L548)

 ___

@@ -46,4 +46,4 @@ Overwrite the existing [Table](../interfaces/Table.md) if presented.

 #### Defined in

-[index.ts:464](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L464)
+[index.ts:550](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L550)
--- a/docs/src/javascript/interfaces/AwsCredentials.md
+++ b/docs/src/javascript/interfaces/AwsCredentials.md
@@ -0,0 +1,41 @@
+[vectordb](../README.md) / [Exports](../modules.md) / AwsCredentials
+
+# Interface: AwsCredentials
+
+## Table of contents
+
+### Properties
+
+- [accessKeyId](AwsCredentials.md#accesskeyid)
+- [secretKey](AwsCredentials.md#secretkey)
+- [sessionToken](AwsCredentials.md#sessiontoken)
+
+## Properties
+
+### accessKeyId
+
+• **accessKeyId**: `string`
+
+#### Defined in
+
+[index.ts:31](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L31)
+
+___
+
+### secretKey
+
+• **secretKey**: `string`
+
+#### Defined in
+
+[index.ts:33](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L33)
+
+___
+
+### sessionToken
+
+• `Optional` **sessionToken**: `string`
+
+#### Defined in
+
+[index.ts:35](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L35)
--- a/docs/src/javascript/interfaces/Connection.md
+++ b/docs/src/javascript/interfaces/Connection.md
@@ -32,7 +32,7 @@ Connection could be local against filesystem or remote against a server.

 #### Defined in

-[index.ts:45](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L45)
+[index.ts:70](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L70)

 ## Methods

@@ -63,7 +63,7 @@ Creates a new Table and initialize it with new data.

 #### Defined in

-[index.ts:65](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L65)
+[index.ts:90](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L90)

 ___

@@ -84,7 +84,7 @@ ___

 #### Defined in

-[index.ts:67](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L67)
+[index.ts:92](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L92)

 ___

@@ -106,7 +106,7 @@ Drop an existing table.

 #### Defined in

-[index.ts:73](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L73)
+[index.ts:98](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L98)

 ___

@@ -135,7 +135,7 @@ Open a table in the database.

 #### Defined in

-[index.ts:55](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L55)
+[index.ts:80](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L80)

 ___

@@ -149,4 +149,4 @@ ___

 #### Defined in

-[index.ts:47](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L47)
+[index.ts:72](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L72)
--- a/docs/src/javascript/interfaces/ConnectionOptions.md
+++ b/docs/src/javascript/interfaces/ConnectionOptions.md
@@ -0,0 +1,30 @@
+[vectordb](../README.md) / [Exports](../modules.md) / ConnectionOptions
+
+# Interface: ConnectionOptions
+
+## Table of contents
+
+### Properties
+
+- [awsCredentials](ConnectionOptions.md#awscredentials)
+- [uri](ConnectionOptions.md#uri)
+
+## Properties
+
+### awsCredentials
+
+• `Optional` **awsCredentials**: [`AwsCredentials`](AwsCredentials.md)
+
+#### Defined in
+
+[index.ts:40](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L40)
+
+___
+
+### uri
+
+• **uri**: `string`
+
+#### Defined in
+
+[index.ts:39](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L39)
--- a/docs/src/javascript/interfaces/EmbeddingFunction.md
+++ b/docs/src/javascript/interfaces/EmbeddingFunction.md
@@ -45,7 +45,7 @@ Creates a vector representation for the given values.

 #### Defined in

-[embedding/embedding_function.ts:27](https://github.com/lancedb/lancedb/blob/7247834/node/src/embedding/embedding_function.ts#L27)
+[embedding/embedding_function.ts:27](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/embedding/embedding_function.ts#L27)

 ___

@@ -57,4 +57,4 @@ The name of the column that will be used as input for the Embedding Function.

 #### Defined in

-[embedding/embedding_function.ts:22](https://github.com/lancedb/lancedb/blob/7247834/node/src/embedding/embedding_function.ts#L22)
+[embedding/embedding_function.ts:22](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/embedding/embedding_function.ts#L22)
--- a/docs/src/javascript/interfaces/IvfPQIndexConfig.md
+++ b/docs/src/javascript/interfaces/IvfPQIndexConfig.md
@@ -0,0 +1,149 @@
+[vectordb](../README.md) / [Exports](../modules.md) / IvfPQIndexConfig
+
+# Interface: IvfPQIndexConfig
+
+## Table of contents
+
+### Properties
+
+- [column](IvfPQIndexConfig.md#column)
+- [index\_name](IvfPQIndexConfig.md#index_name)
+- [max\_iters](IvfPQIndexConfig.md#max_iters)
+- [max\_opq\_iters](IvfPQIndexConfig.md#max_opq_iters)
+- [metric\_type](IvfPQIndexConfig.md#metric_type)
+- [num\_bits](IvfPQIndexConfig.md#num_bits)
+- [num\_partitions](IvfPQIndexConfig.md#num_partitions)
+- [num\_sub\_vectors](IvfPQIndexConfig.md#num_sub_vectors)
+- [replace](IvfPQIndexConfig.md#replace)
+- [type](IvfPQIndexConfig.md#type)
+- [use\_opq](IvfPQIndexConfig.md#use_opq)
+
+## Properties
+
+### column
+
+• `Optional` **column**: `string`
+
+The column to be indexed
+
+#### Defined in
+
+[index.ts:382](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L382)
+
+___
+
+### index\_name
+
+• `Optional` **index\_name**: `string`
+
+A unique name for the index
+
+#### Defined in
+
+[index.ts:387](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L387)
+
+___
+
+### max\_iters
+
+• `Optional` **max\_iters**: `number`
+
+The max number of iterations for kmeans training.
+
+#### Defined in
+
+[index.ts:402](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L402)
+
+___
+
+### max\_opq\_iters
+
+• `Optional` **max\_opq\_iters**: `number`
+
+Max number of iterations to train OPQ, if `use_opq` is true.
+
+#### Defined in
+
+[index.ts:421](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L421)
+
+___
+
+### metric\_type
+
+• `Optional` **metric\_type**: [`MetricType`](../enums/MetricType.md)
+
+Metric type, L2 or Cosine
+
+#### Defined in
+
+[index.ts:392](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L392)
+
+___
+
+### num\_bits
+
+• `Optional` **num\_bits**: `number`
+
+The number of bits to present one PQ centroid.
+
+#### Defined in
+
+[index.ts:416](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L416)
+
+___
+
+### num\_partitions
+
+• `Optional` **num\_partitions**: `number`
+
+The number of partitions this index
+
+#### Defined in
+
+[index.ts:397](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L397)
+
+___
+
+### num\_sub\_vectors
+
+• `Optional` **num\_sub\_vectors**: `number`
+
+Number of subvectors to build PQ code
+
+#### Defined in
+
+[index.ts:412](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L412)
+
+___
+
+### replace
+
+• `Optional` **replace**: `boolean`
+
+Replace an existing index with the same name if it exists.
+
+#### Defined in
+
+[index.ts:426](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L426)
+
+___
+
+### type
+
+• **type**: ``"ivf_pq"``
+
+#### Defined in
+
+[index.ts:428](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L428)
+
+___
+
+### use\_opq
+
+• `Optional` **use\_opq**: `boolean`
+
+Train as optimized product quantization.
+
+#### Defined in
+
+[index.ts:407](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L407)
--- a/docs/src/javascript/interfaces/Table.md
+++ b/docs/src/javascript/interfaces/Table.md
@@ -52,7 +52,7 @@ The number of rows added to the table

 #### Defined in

-[index.ts:95](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L95)
+[index.ts:120](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L120)

 ___

@@ -72,13 +72,13 @@ Returns the number of rows in this table.

 #### Defined in

-[index.ts:115](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L115)
+[index.ts:140](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L140)

 ___

 ### createIndex

-• **createIndex**: (`indexParams`: `IvfPQIndexConfig`) => `Promise`<`any`\>
+• **createIndex**: (`indexParams`: [`IvfPQIndexConfig`](IvfPQIndexConfig.md)) => `Promise`<`any`\>

 #### Type declaration

@@ -94,7 +94,7 @@ VectorIndexParams.

 | Name | Type | Description |
 | :------ | :------ | :------ |
-| `indexParams` | `IvfPQIndexConfig` | The parameters of this Index, |
+| `indexParams` | [`IvfPQIndexConfig`](IvfPQIndexConfig.md) | The parameters of this Index, |

 ##### Returns

@@ -102,7 +102,7 @@ VectorIndexParams.

 #### Defined in

-[index.ts:110](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L110)
+[index.ts:135](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L135)

 ___

@@ -116,11 +116,37 @@ ___

 Delete rows from this table.

+This can be used to delete a single row, many rows, all rows, or
+sometimes no rows (if your predicate matches nothing).
+
+**`Examples`**
+
+```ts
+const con = await lancedb.connect("./.lancedb")
+const data = [
+   {id: 1, vector: [1, 2]},
+   {id: 2, vector: [3, 4]},
+   {id: 3, vector: [5, 6]},
+];
+const tbl = await con.createTable("my_table", data)
+await tbl.delete("id = 2")
+await tbl.countRows() // Returns 2
+```
+
+If you have a list of values to delete, you can combine them into a
+stringified list and use the `IN` operator:
+
+```ts
+const to_remove = [1, 5];
+await tbl.delete(`id IN (${to_remove.join(",")})`)
+await tbl.countRows() // Returns 1
+```
+
 ##### Parameters

 | Name | Type | Description |
 | :------ | :------ | :------ |
-| `filter` | `string` | A filter in the same format used by a sql WHERE clause. |
+| `filter` | `string` | A filter in the same format used by a sql WHERE clause. The filter must not be empty. |

 ##### Returns

@@ -128,7 +154,7 @@ Delete rows from this table.

 #### Defined in

-[index.ts:122](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L122)
+[index.ts:174](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L174)

 ___

@@ -138,7 +164,7 @@ ___

 #### Defined in

-[index.ts:81](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L81)
+[index.ts:106](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L106)

 ___

@@ -166,7 +192,7 @@ The number of rows added to the table

 #### Defined in

-[index.ts:103](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L103)
+[index.ts:128](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L128)

 ___

@@ -192,4 +218,4 @@ Creates a search query to find the nearest neighbors of the given search term

 #### Defined in

-[index.ts:87](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L87)
+[index.ts:112](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L112)
--- a/docs/src/javascript/modules.md
+++ b/docs/src/javascript/modules.md
@@ -18,8 +18,11 @@

 ### Interfaces

+- [AwsCredentials](interfaces/AwsCredentials.md)
 - [Connection](interfaces/Connection.md)
+- [ConnectionOptions](interfaces/ConnectionOptions.md)
 - [EmbeddingFunction](interfaces/EmbeddingFunction.md)
+- [IvfPQIndexConfig](interfaces/IvfPQIndexConfig.md)
 - [Table](interfaces/Table.md)

 ### Type Aliases
@@ -34,11 +37,11 @@

 ### VectorIndexParams

-Ƭ **VectorIndexParams**: `IvfPQIndexConfig`
+Ƭ **VectorIndexParams**: [`IvfPQIndexConfig`](interfaces/IvfPQIndexConfig.md)

 #### Defined in

-[index.ts:345](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L345)
+[index.ts:431](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L431)

 ## Functions

@@ -60,4 +63,20 @@ Connect to a LanceDB instance at the given URI

 #### Defined in

-[index.ts:34](https://github.com/lancedb/lancedb/blob/7247834/node/src/index.ts#L34)
+[index.ts:47](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L47)
+
+▸ **connect**(`opts`): `Promise`<[`Connection`](interfaces/Connection.md)\>
+
+#### Parameters
+
+| Name | Type |
+| :------ | :------ |
+| `opts` | `Partial`<[`ConnectionOptions`](interfaces/ConnectionOptions.md)\> |
+
+#### Returns
+
+`Promise`<[`Connection`](interfaces/Connection.md)\>
+
+#### Defined in
+
+[index.ts:48](https://github.com/lancedb/lancedb/blob/b1eeb90/node/src/index.ts#L48)
--- a/node/package-lock.json
+++ b/node/package-lock.json
@@ -1,12 +1,12 @@
 {
  "name": "vectordb",
-  "version": "0.1.13",
+  "version": "0.1.15",
  "lockfileVersion": 2,
  "requires": true,
  "packages": {
    "": {
      "name": "vectordb",
-      "version": "0.1.13",
+      "version": "0.1.15",
      "cpu": [
        "x64",
        "arm64"
@@ -20,7 +20,8 @@
      "dependencies": {
        "@apache-arrow/ts": "^12.0.0",
        "@neon-rs/load": "^0.0.74",
-        "apache-arrow": "^12.0.0"
+        "apache-arrow": "^12.0.0",
+        "axios": "^1.4.0"
      },
      "devDependencies": {
        "@neon-rs/cli": "^0.0.74",
@@ -50,11 +51,11 @@
        "typescript": "*"
      },
      "optionalDependencies": {
-        "vectordb-darwin-arm64": "0.1.13",
-        "vectordb-darwin-x64": "0.1.13",
-        "vectordb-linux-arm64-gnu": "0.1.13",
-        "vectordb-linux-x64-gnu": "0.1.13",
-        "vectordb-win32-x64-msvc": "0.1.13"
+        "vectordb-darwin-arm64": "0.1.15",
+        "vectordb-darwin-x64": "0.1.15",
+        "vectordb-linux-arm64-gnu": "0.1.15",
+        "vectordb-linux-x64-gnu": "0.1.15",
+        "vectordb-win32-x64-msvc": "0.1.15"
      }
    },
    "node_modules/@apache-arrow/ts": {
@@ -842,8 +843,7 @@
    "node_modules/asynckit": {
      "version": "0.4.0",
      "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
-      "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
-      "dev": true
+      "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q=="
    },
    "node_modules/available-typed-arrays": {
      "version": "1.0.5",
@@ -858,12 +858,13 @@
      }
    },
    "node_modules/axios": {
-      "version": "0.26.1",
-      "resolved": "https://registry.npmjs.org/axios/-/axios-0.26.1.tgz",
-      "integrity": "sha512-fPwcX4EvnSHuInCMItEhAGnaSEXRBjtzh9fOtsE6E1G6p7vl7edEeZe11QHf18+6+9gR5PbKV/sGKNaD8YaMeA==",
-      "dev": true,
+      "version": "1.4.0",
+      "resolved": "https://registry.npmjs.org/axios/-/axios-1.4.0.tgz",
+      "integrity": "sha512-S4XCWMEmzvo64T9GfvQDOXgYRDJ/wsSZc7Jvdgx5u1sd0JwsuPLqb3SYmusag+edF6ziyMensPVqLTSc1PiSEA==",
      "dependencies": {
-        "follow-redirects": "^1.14.8"
+        "follow-redirects": "^1.15.0",
+        "form-data": "^4.0.0",
+        "proxy-from-env": "^1.1.0"
      }
    },
    "node_modules/balanced-match": {
@@ -1094,7 +1095,6 @@
      "version": "1.0.8",
      "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
      "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
-      "dev": true,
      "dependencies": {
        "delayed-stream": "~1.0.0"
      },
@@ -1317,7 +1317,6 @@
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
      "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
-      "dev": true,
      "engines": {
        "node": ">=0.4.0"
      }
@@ -2084,7 +2083,6 @@
      "version": "1.15.2",
      "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.2.tgz",
      "integrity": "sha512-VQLG33o04KaQ8uYi2tVNbdrWp1QWxNNea+nmIB4EVM28v0hmP17z7aG1+wAkNzVq4KeXTq3221ye5qTJP91JwA==",
-      "dev": true,
      "funding": [
        {
          "type": "individual",
@@ -2113,7 +2111,6 @@
      "version": "4.0.0",
      "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.0.tgz",
      "integrity": "sha512-ETEklSGi5t0QMZuiXoA/Q6vcnxcLQP5vdugSpuAyi6SVGi2clPPp+xgEhuMaHC+zGgn31Kd235W35f7Hykkaww==",
-      "dev": true,
      "dependencies": {
        "asynckit": "^0.4.0",
        "combined-stream": "^1.0.8",
@@ -2987,7 +2984,6 @@
      "version": "1.52.0",
      "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
      "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
-      "dev": true,
      "engines": {
        "node": ">= 0.6"
      }
@@ -2996,7 +2992,6 @@
      "version": "2.1.35",
      "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
      "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
-      "dev": true,
      "dependencies": {
        "mime-db": "1.52.0"
      },
@@ -3290,6 +3285,15 @@
        "form-data": "^4.0.0"
      }
    },
+    "node_modules/openai/node_modules/axios": {
+      "version": "0.26.1",
+      "resolved": "https://registry.npmjs.org/axios/-/axios-0.26.1.tgz",
+      "integrity": "sha512-fPwcX4EvnSHuInCMItEhAGnaSEXRBjtzh9fOtsE6E1G6p7vl7edEeZe11QHf18+6+9gR5PbKV/sGKNaD8YaMeA==",
+      "dev": true,
+      "dependencies": {
+        "follow-redirects": "^1.14.8"
+      }
+    },
    "node_modules/optionator": {
      "version": "0.9.1",
      "resolved": "https://registry.npmjs.org/optionator/-/optionator-0.9.1.tgz",
@@ -3441,6 +3445,11 @@
        "node": ">= 0.8.0"
      }
    },
+    "node_modules/proxy-from-env": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-1.1.0.tgz",
+      "integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg=="
+    },
    "node_modules/punycode": {
      "version": "2.3.0",
      "resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.0.tgz",
@@ -4288,42 +4297,6 @@
      "integrity": "sha512-wa7YjyUGfNZngI/vtK0UHAN+lgDCxBPCylVXGp0zu59Fz5aiGtNXaq3DhIov063MorB+VfufLh3JlF2KdTK3xg==",
      "dev": true
    },
-    "node_modules/vectordb-darwin-arm64": {
-      "version": "0.1.13",
-      "resolved": "https://registry.npmjs.org/vectordb-darwin-arm64/-/vectordb-darwin-arm64-0.1.13.tgz",
-      "integrity": "sha512-9lLuX5P8m75EfP85pfC4LxO9J7Tzu4LngX55BVAdFe6qPRHu+iHmLw0QYYSVDqNm3GtDr2qFJlL2ILlsApyYyg==",
-      "cpu": [
-        "arm64"
-      ],
-      "optional": true,
-      "os": [
-        "darwin"
-      ]
-    },
-    "node_modules/vectordb-darwin-x64": {
-      "version": "0.1.13",
-      "resolved": "https://registry.npmjs.org/vectordb-darwin-x64/-/vectordb-darwin-x64-0.1.13.tgz",
-      "integrity": "sha512-5mkhBJlcfAqcty7Ww2csgYogq+b0NhtllAbag9IIznvqfcrvITU0H0vm5LGWbRuE/BUUxC25MJhm93YWBzqEVA==",
-      "cpu": [
-        "x64"
-      ],
-      "optional": true,
-      "os": [
-        "darwin"
-      ]
-    },
-    "node_modules/vectordb-linux-x64-gnu": {
-      "version": "0.1.13",
-      "resolved": "https://registry.npmjs.org/vectordb-linux-x64-gnu/-/vectordb-linux-x64-gnu-0.1.13.tgz",
-      "integrity": "sha512-fU+sIHUkXyMdrWjggT93p0blKD+pbgr+x01tn9d2/pbA1ePo2AwuE86rYPA+BjyCUE1QifPgKadzGVVpqWYmnQ==",
-      "cpu": [
-        "x64"
-      ],
-      "optional": true,
-      "os": [
-        "linux"
-      ]
-    },
    "node_modules/vscode-oniguruma": {
      "version": "1.7.0",
      "resolved": "https://registry.npmjs.org/vscode-oniguruma/-/vscode-oniguruma-1.7.0.tgz",
@@ -5135,8 +5108,7 @@
    "asynckit": {
      "version": "0.4.0",
      "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
-      "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
-      "dev": true
+      "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q=="
    },
    "available-typed-arrays": {
      "version": "1.0.5",
@@ -5145,12 +5117,13 @@
      "dev": true
    },
    "axios": {
-      "version": "0.26.1",
-      "resolved": "https://registry.npmjs.org/axios/-/axios-0.26.1.tgz",
-      "integrity": "sha512-fPwcX4EvnSHuInCMItEhAGnaSEXRBjtzh9fOtsE6E1G6p7vl7edEeZe11QHf18+6+9gR5PbKV/sGKNaD8YaMeA==",
-      "dev": true,
+      "version": "1.4.0",
+      "resolved": "https://registry.npmjs.org/axios/-/axios-1.4.0.tgz",
+      "integrity": "sha512-S4XCWMEmzvo64T9GfvQDOXgYRDJ/wsSZc7Jvdgx5u1sd0JwsuPLqb3SYmusag+edF6ziyMensPVqLTSc1PiSEA==",
      "requires": {
-        "follow-redirects": "^1.14.8"
+        "follow-redirects": "^1.15.0",
+        "form-data": "^4.0.0",
+        "proxy-from-env": "^1.1.0"
      }
    },
    "balanced-match": {
@@ -5330,7 +5303,6 @@
      "version": "1.0.8",
      "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
      "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
-      "dev": true,
      "requires": {
        "delayed-stream": "~1.0.0"
      }
@@ -5497,8 +5469,7 @@
    "delayed-stream": {
      "version": "1.0.0",
      "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
-      "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
-      "dev": true
+      "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ=="
    },
    "diff": {
      "version": "4.0.2",
@@ -6068,8 +6039,7 @@
    "follow-redirects": {
      "version": "1.15.2",
      "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.2.tgz",
-      "integrity": "sha512-VQLG33o04KaQ8uYi2tVNbdrWp1QWxNNea+nmIB4EVM28v0hmP17z7aG1+wAkNzVq4KeXTq3221ye5qTJP91JwA==",
-      "dev": true
+      "integrity": "sha512-VQLG33o04KaQ8uYi2tVNbdrWp1QWxNNea+nmIB4EVM28v0hmP17z7aG1+wAkNzVq4KeXTq3221ye5qTJP91JwA=="
    },
    "for-each": {
      "version": "0.3.3",
@@ -6084,7 +6054,6 @@
      "version": "4.0.0",
      "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.0.tgz",
      "integrity": "sha512-ETEklSGi5t0QMZuiXoA/Q6vcnxcLQP5vdugSpuAyi6SVGi2clPPp+xgEhuMaHC+zGgn31Kd235W35f7Hykkaww==",
-      "dev": true,
      "requires": {
        "asynckit": "^0.4.0",
        "combined-stream": "^1.0.8",
@@ -6698,14 +6667,12 @@
    "mime-db": {
      "version": "1.52.0",
      "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
-      "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
-      "dev": true
+      "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg=="
    },
    "mime-types": {
      "version": "2.1.35",
      "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
      "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
-      "dev": true,
      "requires": {
        "mime-db": "1.52.0"
      }
@@ -6931,6 +6898,17 @@
      "requires": {
        "axios": "^0.26.0",
        "form-data": "^4.0.0"
+      },
+      "dependencies": {
+        "axios": {
+          "version": "0.26.1",
+          "resolved": "https://registry.npmjs.org/axios/-/axios-0.26.1.tgz",
+          "integrity": "sha512-fPwcX4EvnSHuInCMItEhAGnaSEXRBjtzh9fOtsE6E1G6p7vl7edEeZe11QHf18+6+9gR5PbKV/sGKNaD8YaMeA==",
+          "dev": true,
+          "requires": {
+            "follow-redirects": "^1.14.8"
+          }
+        }
      }
    },
    "optionator": {
@@ -7039,6 +7017,11 @@
      "integrity": "sha512-vkcDPrRZo1QZLbn5RLGPpg/WmIQ65qoWWhcGKf/b5eplkkarX0m9z8ppCat4mlOqUsWpyNuYgO3VRyrYHSzX5g==",
      "dev": true
    },
+    "proxy-from-env": {
+      "version": "1.1.0",
+      "resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-1.1.0.tgz",
+      "integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg=="
+    },
    "punycode": {
      "version": "2.3.0",
      "resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.0.tgz",
@@ -7619,24 +7602,6 @@
      "integrity": "sha512-wa7YjyUGfNZngI/vtK0UHAN+lgDCxBPCylVXGp0zu59Fz5aiGtNXaq3DhIov063MorB+VfufLh3JlF2KdTK3xg==",
      "dev": true
    },
-    "vectordb-darwin-arm64": {
-      "version": "0.1.13",
-      "resolved": "https://registry.npmjs.org/vectordb-darwin-arm64/-/vectordb-darwin-arm64-0.1.13.tgz",
-      "integrity": "sha512-9lLuX5P8m75EfP85pfC4LxO9J7Tzu4LngX55BVAdFe6qPRHu+iHmLw0QYYSVDqNm3GtDr2qFJlL2ILlsApyYyg==",
-      "optional": true
-    },
-    "vectordb-darwin-x64": {
-      "version": "0.1.13",
-      "resolved": "https://registry.npmjs.org/vectordb-darwin-x64/-/vectordb-darwin-x64-0.1.13.tgz",
-      "integrity": "sha512-5mkhBJlcfAqcty7Ww2csgYogq+b0NhtllAbag9IIznvqfcrvITU0H0vm5LGWbRuE/BUUxC25MJhm93YWBzqEVA==",
-      "optional": true
-    },
-    "vectordb-linux-x64-gnu": {
-      "version": "0.1.13",
-      "resolved": "https://registry.npmjs.org/vectordb-linux-x64-gnu/-/vectordb-linux-x64-gnu-0.1.13.tgz",
-      "integrity": "sha512-fU+sIHUkXyMdrWjggT93p0blKD+pbgr+x01tn9d2/pbA1ePo2AwuE86rYPA+BjyCUE1QifPgKadzGVVpqWYmnQ==",
-      "optional": true
-    },
    "vscode-oniguruma": {
      "version": "1.7.0",
      "resolved": "https://registry.npmjs.org/vscode-oniguruma/-/vscode-oniguruma-1.7.0.tgz",
--- a/node/package.json
+++ b/node/package.json
@@ -1,6 +1,6 @@
 {
  "name": "vectordb",
-  "version": "0.1.13",
+  "version": "0.1.15",
  "description": " Serverless, low-latency vector database for AI applications",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
@@ -56,7 +56,8 @@
  "dependencies": {
    "@apache-arrow/ts": "^12.0.0",
    "@neon-rs/load": "^0.0.74",
-    "apache-arrow": "^12.0.0"
+    "apache-arrow": "^12.0.0",
+    "axios": "^1.4.0"
  },
  "os": [
    "darwin",
@@ -77,10 +78,10 @@
    }
  },
  "optionalDependencies": {
-    "vectordb-darwin-arm64": "0.1.13",
-    "vectordb-darwin-x64": "0.1.13",
-    "vectordb-linux-arm64-gnu": "0.1.13",
-    "vectordb-linux-x64-gnu": "0.1.13",
-    "vectordb-win32-x64-msvc": "0.1.13"
+    "vectordb-darwin-arm64": "0.1.15",
+    "vectordb-darwin-x64": "0.1.15",
+    "vectordb-linux-arm64-gnu": "0.1.15",
+    "vectordb-linux-x64-gnu": "0.1.15",
+    "vectordb-win32-x64-msvc": "0.1.15"
  }
 }
--- a/node/src/index.ts
+++ b/node/src/index.ts
@@ -14,15 +14,15 @@

 import {
  RecordBatchFileWriter,
-  type Table as ArrowTable,
-  tableFromIPC,
-  Vector
+  type Table as ArrowTable
 } from 'apache-arrow'
 import { fromRecordsToBuffer } from './arrow'
 import type { EmbeddingFunction } from './embedding/embedding_function'
+import { RemoteConnection } from './remote'
+import { Query } from './query'

 // eslint-disable-next-line @typescript-eslint/no-var-requires
-const { databaseNew, databaseTableNames, databaseOpenTable, databaseDropTable, tableCreate, tableSearch, tableAdd, tableCreateVectorIndex, tableCountRows, tableDelete } = require('../native.js')
+const { databaseNew, databaseTableNames, databaseOpenTable, databaseDropTable, tableCreate, tableAdd, tableCreateVectorIndex, tableCountRows, tableDelete } = require('../native.js')

 export type { EmbeddingFunction }
 export { OpenAIEmbeddingFunction } from './embedding/openai'
@@ -37,7 +37,16 @@ export interface AwsCredentials {

 export interface ConnectionOptions {
  uri: string
+
  awsCredentials?: AwsCredentials
+
+  // API key for the remote connections
+  apiKey?: string
+  // Region to connect
+  region?: string
+
+  // override the host for the remote connections
+  hostOverride?: string
 }

 /**
@@ -54,9 +63,16 @@ export async function connect (arg: string | Partial<ConnectionOptions>): Promis
    // opts = { uri: arg.uri, awsCredentials = arg.awsCredentials }
    opts = Object.assign({
      uri: '',
-      awsCredentials: undefined
+      awsCredentials: undefined,
+      apiKey: undefined,
+      region: 'us-west-2'
    }, arg)
  }
+
+  if (opts.uri.startsWith('db://')) {
+    // Remote connection
+    return new RemoteConnection(opts)
+  }
  const db = await databaseNew(opts.uri)
  return new LocalConnection(db, opts)
 }
@@ -142,7 +158,34 @@ export interface Table<T = number[]> {
  /**
   * Delete rows from this table.
   *
-   * @param filter  A filter in the same format used by a sql WHERE clause.
+   * This can be used to delete a single row, many rows, all rows, or
+   * sometimes no rows (if your predicate matches nothing).
+   *
+   * @param filter  A filter in the same format used by a sql WHERE clause. The
+   *                filter must not be empty.
+   *
+   * @examples
+   *
+   * ```ts
+   * const con = await lancedb.connect("./.lancedb")
+   * const data = [
+   *    {id: 1, vector: [1, 2]},
+   *    {id: 2, vector: [3, 4]},
+   *    {id: 3, vector: [5, 6]},
+   * ];
+   * const tbl = await con.createTable("my_table", data)
+   * await tbl.delete("id = 2")
+   * await tbl.countRows() // Returns 2
+   * ```
+   *
+   * If you have a list of values to delete, you can combine them into a
+   * stringified list and use the `IN` operator:
+   *
+   * ```ts
+   * const to_remove = [1, 5];
+   * await tbl.delete(`id IN (${to_remove.join(",")})`)
+   * await tbl.countRows() // Returns 1
+   * ```
   */
  delete: (filter: string) => Promise<void>
 }
@@ -164,8 +207,8 @@ export class LocalConnection implements Connection {
  }

  /**
-     * Get the names of all tables in the database.
-     */
+   * Get the names of all tables in the database.
+   */
  async tableNames (): Promise<string[]> {
    return databaseTableNames.call(this._db)
  }
@@ -176,6 +219,7 @@ export class LocalConnection implements Connection {
   * @param name The name of the table.
   */
  async openTable (name: string): Promise<Table>
+
  /**
   * Open a table in the database.
   *
@@ -281,7 +325,7 @@ export class LocalTable<T = number[]> implements Table<T> {
   * @param query The query search term
   */
  search (query: T): Query<T> {
-    return new Query(this._tbl, query, this._embeddings)
+    return new Query(query, this._tbl, this._embeddings)
  }

  /**
@@ -403,116 +447,6 @@ export interface IvfPQIndexConfig {

 export type VectorIndexParams = IvfPQIndexConfig

-/**
- * A builder for nearest neighbor queries for LanceDB.
- */
-export class Query<T = number[]> {
-  private readonly _tbl: any
-  private readonly _query: T
-  private _queryVector?: number[]
-  private _limit: number
-  private _refineFactor?: number
-  private _nprobes: number
-  private _select?: string[]
-  private _filter?: string
-  private _metricType?: MetricType
-  private readonly _embeddings?: EmbeddingFunction<T>
-
-  constructor (tbl: any, query: T, embeddings?: EmbeddingFunction<T>) {
-    this._tbl = tbl
-    this._query = query
-    this._limit = 10
-    this._nprobes = 20
-    this._refineFactor = undefined
-    this._select = undefined
-    this._filter = undefined
-    this._metricType = undefined
-    this._embeddings = embeddings
-  }
-
-  /***
-   * Sets the number of results that will be returned
-   * @param value number of results
-   */
-  limit (value: number): Query<T> {
-    this._limit = value
-    return this
-  }
-
-  /**
-   * Refine the results by reading extra elements and re-ranking them in memory.
-   * @param value refine factor to use in this query.
-   */
-  refineFactor (value: number): Query<T> {
-    this._refineFactor = value
-    return this
-  }
-
-  /**
-   * The number of probes used. A higher number makes search more accurate but also slower.
-   * @param value The number of probes used.
-   */
-  nprobes (value: number): Query<T> {
-    this._nprobes = value
-    return this
-  }
-
-  /**
-   * A filter statement to be applied to this query.
-   * @param value A filter in the same format used by a sql WHERE clause.
-   */
-  filter (value: string): Query<T> {
-    this._filter = value
-    return this
-  }
-
-  where = this.filter
-
-  /** Return only the specified columns.
-   *
-   * @param value Only select the specified columns. If not specified, all columns will be returned.
-   */
-  select (value: string[]): Query<T> {
-    this._select = value
-    return this
-  }
-
-  /**
-   * The MetricType used for this Query.
-   * @param value The metric to the. @see MetricType for the different options
-   */
-  metricType (value: MetricType): Query<T> {
-    this._metricType = value
-    return this
-  }
-
-  /**
-   * Execute the query and return the results as an Array of Objects
-   */
-  async execute<T = Record<string, unknown>> (): Promise<T[]> {
-    if (this._embeddings !== undefined) {
-      this._queryVector = (await this._embeddings.embed([this._query]))[0]
-    } else {
-      this._queryVector = this._query as number[]
-    }
-
-    const buffer = await tableSearch.call(this._tbl, this)
-    const data = tableFromIPC(buffer)
-
-    return data.toArray().map((entry: Record<string, unknown>) => {
-      const newObject: Record<string, unknown> = {}
-      Object.keys(entry).forEach((key: string) => {
-        if (entry[key] instanceof Vector) {
-          newObject[key] = (entry[key] as Vector).toArray()
-        } else {
-          newObject[key] = entry[key]
-        }
-      })
-      return newObject as unknown as T
-    })
-  }
-}
-
 /**
 * Write mode for writing a table.
 */
--- a/node/src/query.ts
+++ b/node/src/query.ts
@@ -0,0 +1,130 @@
+// Copyright 2023 LanceDB Developers.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+import { Vector, tableFromIPC } from 'apache-arrow'
+import { type EmbeddingFunction } from './embedding/embedding_function'
+import { type MetricType } from '.'
+
+// eslint-disable-next-line @typescript-eslint/no-var-requires
+const { tableSearch } = require('../native.js')
+
+/**
+ * A builder for nearest neighbor queries for LanceDB.
+ */
+export class Query<T = number[]> {
+  private readonly _query: T
+  private readonly _tbl?: any
+  private _queryVector?: number[]
+  private _limit: number
+  private _refineFactor?: number
+  private _nprobes: number
+  private _select?: string[]
+  private _filter?: string
+  private _metricType?: MetricType
+  protected readonly _embeddings?: EmbeddingFunction<T>
+
+  constructor (query: T, tbl?: any, embeddings?: EmbeddingFunction<T>) {
+    this._tbl = tbl
+    this._query = query
+    this._limit = 10
+    this._nprobes = 20
+    this._refineFactor = undefined
+    this._select = undefined
+    this._filter = undefined
+    this._metricType = undefined
+    this._embeddings = embeddings
+  }
+
+  /***
+     * Sets the number of results that will be returned
+     * @param value number of results
+     */
+  limit (value: number): Query<T> {
+    this._limit = value
+    return this
+  }
+
+  /**
+     * Refine the results by reading extra elements and re-ranking them in memory.
+     * @param value refine factor to use in this query.
+     */
+  refineFactor (value: number): Query<T> {
+    this._refineFactor = value
+    return this
+  }
+
+  /**
+     * The number of probes used. A higher number makes search more accurate but also slower.
+     * @param value The number of probes used.
+     */
+  nprobes (value: number): Query<T> {
+    this._nprobes = value
+    return this
+  }
+
+  /**
+     * A filter statement to be applied to this query.
+     * @param value A filter in the same format used by a sql WHERE clause.
+     */
+  filter (value: string): Query<T> {
+    this._filter = value
+    return this
+  }
+
+  where = this.filter
+
+  /** Return only the specified columns.
+     *
+     * @param value Only select the specified columns. If not specified, all columns will be returned.
+     */
+  select (value: string[]): Query<T> {
+    this._select = value
+    return this
+  }
+
+  /**
+     * The MetricType used for this Query.
+     * @param value The metric to the. @see MetricType for the different options
+     */
+  metricType (value: MetricType): Query<T> {
+    this._metricType = value
+    return this
+  }
+
+  /**
+     * Execute the query and return the results as an Array of Objects
+     */
+  async execute<T = Record<string, unknown>> (): Promise<T[]> {
+    if (this._embeddings !== undefined) {
+      this._queryVector = (await this._embeddings.embed([this._query]))[0]
+    } else {
+      this._queryVector = this._query as number[]
+    }
+
+    const buffer = await tableSearch.call(this._tbl, this)
+    const data = tableFromIPC(buffer)
+
+    return data.toArray().map((entry: Record<string, unknown>) => {
+      const newObject: Record<string, unknown> = {}
+      Object.keys(entry).forEach((key: string) => {
+        if (entry[key] instanceof Vector) {
+          newObject[key] = (entry[key] as Vector).toArray()
+        } else {
+          newObject[key] = entry[key]
+        }
+      })
+      return newObject as unknown as T
+    })
+  }
+}
--- a/node/src/remote/client.ts
+++ b/node/src/remote/client.ts
@@ -0,0 +1,105 @@
+// Copyright 2023 LanceDB Developers.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+import axios, { type AxiosResponse } from 'axios'
+
+import { tableFromIPC, type Table as ArrowTable } from 'apache-arrow'
+
+export class HttpLancedbClient {
+  private readonly _url: string
+
+  public constructor (
+    url: string,
+    private readonly _apiKey: string,
+    private readonly _dbName?: string
+  ) {
+    this._url = url
+  }
+
+  get uri (): string {
+    return this._url
+  }
+
+  public async search (
+    tableName: string,
+    vector: number[],
+    k: number,
+    nprobes: number,
+    refineFactor?: number,
+    columns?: string[],
+    filter?: string
+  ): Promise<ArrowTable<any>> {
+    const response = await axios.post(
+              `${this._url}/v1/table/${tableName}`,
+              {
+                vector,
+                k,
+                nprobes,
+                refineFactor,
+                columns,
+                filter
+              },
+              {
+                headers: {
+                  'Content-Type': 'application/json',
+                  'x-api-key': this._apiKey,
+                  ...(this._dbName !== undefined ? { 'x-lancedb-database': this._dbName } : {})
+                },
+                responseType: 'arraybuffer',
+                timeout: 10000
+              }
+    ).catch((err) => {
+      console.error('error: ', err)
+      return err.response
+    })
+    if (response.status !== 200) {
+      const errorData = new TextDecoder().decode(response.data)
+      throw new Error(
+        `Server Error, status: ${response.status as number}, ` +
+        `message: ${response.statusText as string}: ${errorData}`
+      )
+    }
+
+    const table = tableFromIPC(response.data)
+    return table
+  }
+
+  /**
+   * Sent GET request.
+   */
+  public async get (path: string, params?: Record<string, string | number>): Promise<AxiosResponse> {
+    const response = await axios.get(
+      `${this._url}${path}`,
+      {
+        headers: {
+          'Content-Type': 'application/json',
+          'x-api-key': this._apiKey
+        },
+        params,
+        timeout: 10000
+      }
+    ).catch((err) => {
+      console.error('error: ', err)
+      return err.response
+    })
+    if (response.status !== 200) {
+      const errorData = new TextDecoder().decode(response.data)
+      throw new Error(
+        `Server Error, status: ${response.status as number}, ` +
+        `message: ${response.statusText as string}: ${errorData}`
+      )
+    }
+    return response
+  }
+}
--- a/node/src/remote/index.ts
+++ b/node/src/remote/index.ts
@@ -0,0 +1,168 @@
+// Copyright 2023 LanceDB Developers.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+import {
+  type EmbeddingFunction, type Table, type VectorIndexParams, type Connection,
+  type ConnectionOptions
+} from '../index'
+import { Query } from '../query'
+
+import { type Table as ArrowTable, Vector } from 'apache-arrow'
+import { HttpLancedbClient } from './client'
+
+/**
+ * Remote connection.
+ */
+export class RemoteConnection implements Connection {
+  private readonly _client: HttpLancedbClient
+  private readonly _dbName: string
+
+  constructor (opts: ConnectionOptions) {
+    if (!opts.uri.startsWith('db://')) {
+      throw new Error(`Invalid remote DB URI: ${opts.uri}`)
+    }
+    if (opts.apiKey === undefined || opts.region === undefined) {
+      throw new Error('API key and region are not supported for remote connections')
+    }
+
+    this._dbName = opts.uri.slice('db://'.length)
+    let server: string
+    if (opts.hostOverride === undefined) {
+      server = `https://${this._dbName}.${opts.region}.api.lancedb.com`
+    } else {
+      server = opts.hostOverride
+    }
+    this._client = new HttpLancedbClient(server, opts.apiKey, opts.hostOverride === undefined ? undefined : this._dbName)
+  }
+
+  get uri (): string {
+    // add the lancedb+ prefix back
+    return 'db://' + this._client.uri
+  }
+
+  async tableNames (): Promise<string[]> {
+    const response = await this._client.get('/v1/table/')
+    return response.data.tables
+  }
+
+  async openTable (name: string): Promise<Table>
+  async openTable<T> (name: string, embeddings: EmbeddingFunction<T>): Promise<Table<T>>
+  async openTable<T> (name: string, embeddings?: EmbeddingFunction<T>): Promise<Table<T>> {
+    if (embeddings !== undefined) {
+      return new RemoteTable(this._client, name, embeddings)
+    } else {
+      return new RemoteTable(this._client, name)
+    }
+  }
+
+  async createTable (name: string, data: Array<Record<string, unknown>>): Promise<Table>
+  async createTable<T> (name: string, data: Array<Record<string, unknown>>, embeddings: EmbeddingFunction<T>): Promise<Table<T>>
+  async createTable<T> (name: string, data: Array<Record<string, unknown>>, embeddings?: EmbeddingFunction<T>): Promise<Table<T>> {
+    throw new Error('Not implemented')
+  }
+
+  async createTableArrow (name: string, table: ArrowTable): Promise<Table> {
+    throw new Error('Not implemented')
+  }
+
+  async dropTable (name: string): Promise<void> {
+    throw new Error('Not implemented')
+  }
+}
+
+export class RemoteQuery<T = number[]> extends Query<T> {
+  constructor (query: T, private readonly _client: HttpLancedbClient,
+    private readonly _name: string, embeddings?: EmbeddingFunction<T>) {
+    super(query, undefined, embeddings)
+  }
+
+  // TODO: refactor this to a base class + queryImpl pattern
+  async execute<T = Record<string, unknown>>(): Promise<T[]> {
+    const embeddings = this._embeddings
+    const query = (this as any)._query
+    let queryVector: number[]
+
+    if (embeddings !== undefined) {
+      queryVector = (await embeddings.embed([query]))[0]
+    } else {
+      queryVector = query as number[]
+    }
+
+    const data = await this._client.search(
+      this._name,
+      queryVector,
+      (this as any)._limit,
+      (this as any)._nprobes,
+      (this as any)._refineFactor,
+      (this as any)._select,
+      (this as any)._filter
+    )
+
+    return data.toArray().map((entry: Record<string, unknown>) => {
+      const newObject: Record<string, unknown> = {}
+      Object.keys(entry).forEach((key: string) => {
+        if (entry[key] instanceof Vector) {
+          newObject[key] = (entry[key] as Vector).toArray()
+        } else {
+          newObject[key] = entry[key]
+        }
+      })
+      return newObject as unknown as T
+    })
+  }
+}
+
+// we are using extend until we have next next version release
+// Table and Connection has both been refactored to interfaces
+export class RemoteTable<T = number[]> implements Table<T> {
+  private readonly _client: HttpLancedbClient
+  private readonly _embeddings?: EmbeddingFunction<T>
+  private readonly _name: string
+
+  constructor (client: HttpLancedbClient, name: string)
+  constructor (client: HttpLancedbClient, name: string, embeddings: EmbeddingFunction<T>)
+  constructor (client: HttpLancedbClient, name: string, embeddings?: EmbeddingFunction<T>) {
+    this._client = client
+    this._name = name
+    this._embeddings = embeddings
+  }
+
+  get name (): string {
+    return this._name
+  }
+
+  search (query: T): Query<T> {
+    return new RemoteQuery(query, this._client, this._name)//, this._embeddings_new)
+  }
+
+  async add (data: Array<Record<string, unknown>>): Promise<number> {
+    throw new Error('Not implemented')
+  }
+
+  async overwrite (data: Array<Record<string, unknown>>): Promise<number> {
+    throw new Error('Not implemented')
+  }
+
+  async createIndex (indexParams: VectorIndexParams): Promise<any> {
+    throw new Error('Not implemented')
+  }
+
+  async countRows (): Promise<number> {
+    throw new Error('Not implemented')
+  }
+
+  async delete (filter: string): Promise<void> {
+    throw new Error('Not implemented')
+  }
+}
--- a/node/src/test/test.ts
+++ b/node/src/test/test.ts
@@ -18,7 +18,8 @@ import * as chai from 'chai'
 import * as chaiAsPromised from 'chai-as-promised'

 import * as lancedb from '../index'
-import { type AwsCredentials, type EmbeddingFunction, MetricType, Query, WriteMode } from '../index'
+import { type AwsCredentials, type EmbeddingFunction, MetricType, WriteMode } from '../index'
+import { Query } from '../query'

 const expect = chai.expect
 const assert = chai.assert
@@ -268,7 +269,7 @@ describe('LanceDB client', function () {

 describe('Query object', function () {
  it('sets custom parameters', async function () {
-    const query = new Query(undefined, [0.1, 0.3])
+    const query = new Query([0.1, 0.3])
      .limit(1)
      .metricType(MetricType.Cosine)
      .refineFactor(100)
--- a/python/.bumpversion.cfg
+++ b/python/.bumpversion.cfg
@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 0.1.8
+current_version = 0.1.13
 commit = True
 message = [python] Bump version: {current_version} → {new_version}
 tag = True
--- a/python/lancedb/db.py
+++ b/python/lancedb/db.py
@@ -24,7 +24,7 @@ from pyarrow import fs

 from .common import DATA, URI
 from .table import LanceTable, Table
-from .util import get_uri_location, get_uri_scheme
+from .util import fs_from_uri, get_uri_location, get_uri_scheme


 class DBConnection(ABC):
@@ -252,7 +252,7 @@ class LanceDBConnection(DBConnection):
            A list of table names.
        """
        try:
-            filesystem, path = fs.FileSystem.from_uri(self.uri)
+            filesystem, path = fs_from_uri(self.uri)
        except pa.ArrowInvalid:
            raise NotImplementedError("Unsupported scheme: " + self.uri)

--- a/python/lancedb/pydantic.py
+++ b/python/lancedb/pydantic.py
@@ -11,7 +11,7 @@
 #  See the License for the specific language governing permissions and
 #  limitations under the License.

-"""Pydantic adapter for LanceDB"""
+"""Pydantic (v1 / v2) adapter for LanceDB"""

 from __future__ import annotations

@@ -19,11 +19,19 @@ import inspect
 import sys
 import types
 from abc import ABC, abstractmethod
-from typing import Any, List, Type, Union, _GenericAlias
+from typing import Any, Callable, Dict, Generator, List, Type, Union, _GenericAlias

+import numpy as np
 import pyarrow as pa
 import pydantic
-from pydantic_core import CoreSchema, core_schema
+import semver
+
+PYDANTIC_VERSION = semver.Version.parse(pydantic.__version__)
+try:
+    from pydantic_core import CoreSchema, core_schema
+except ImportError:
+    if PYDANTIC_VERSION >= (2,):
+        raise


 class FixedSizeListMixin(ABC):
@@ -73,6 +81,9 @@ def vector(

    # TODO: make a public parameterized type.
    class FixedSizeList(list, FixedSizeListMixin):
+        def __repr__(self):
+            return f"FixedSizeList(dim={dim})"
+
        @staticmethod
        def dim() -> int:
            return dim
@@ -94,6 +105,25 @@ def vector(
                ),
            )

+        @classmethod
+        def __get_validators__(cls) -> Generator[Callable, None, None]:
+            yield cls.validate
+
+        # For pydantic v1
+        @classmethod
+        def validate(cls, v):
+            if not isinstance(v, (list, range, np.ndarray)) or len(v) != dim:
+                raise TypeError("A list of numbers or numpy.ndarray is needed")
+            return v
+
+        if PYDANTIC_VERSION < (2, 0):
+
+            @classmethod
+            def __modify_schema__(cls, field_schema: Dict[str, Any]):
+                field_schema["items"] = {"type": "number"}
+                field_schema["maxItems"] = dim
+                field_schema["minItems"] = dim
+
    return FixedSizeList


@@ -120,11 +150,20 @@ def _py_type_to_arrow_type(py_type: Type[Any]) -> pa.DataType:
    )


-def _pydantic_model_to_fields(model: pydantic.BaseModel) -> List[pa.Field]:
-    fields = []
-    for name, field in model.model_fields.items():
-        fields.append(_pydantic_to_field(name, field))
-    return fields
+if PYDANTIC_VERSION.major < 2:
+
+    def _pydantic_model_to_fields(model: pydantic.BaseModel) -> List[pa.Field]:
+        return [
+            _pydantic_to_field(name, field) for name, field in model.__fields__.items()
+        ]
+
+else:
+
+    def _pydantic_model_to_fields(model: pydantic.BaseModel) -> List[pa.Field]:
+        return [
+            _pydantic_to_field(name, field)
+            for name, field in model.model_fields.items()
+        ]


 def _pydantic_to_arrow_type(field: pydantic.fields.FieldInfo) -> pa.DataType:
--- a/python/lancedb/remote/table.py
+++ b/python/lancedb/remote/table.py
@@ -79,7 +79,6 @@ class RemoteTable(Table):
                content_type=ARROW_STREAM_CONTENT_TYPE,
            )
        )
-        return len(data)

    def search(
        self, query: Union[VEC, str], vector_column: str = VECTOR_COLUMN_NAME
@@ -89,3 +88,6 @@ class RemoteTable(Table):
    def _execute_query(self, query: Query) -> pa.Table:
        result = self._conn._client.query(self._name, query)
        return self._conn._loop.run_until_complete(result).to_arrow()
+
+    def delete(self, predicate: str):
+        raise NotImplementedError
--- a/python/lancedb/table.py
+++ b/python/lancedb/table.py
@@ -74,7 +74,6 @@ class Table(ABC):
    Can append new data with [Table.add()][lancedb.table.Table.add].

    >>> table.add([{"vector": [0.5, 1.3], "b": 4}])
-    2

    Can query the table with [Table.search][lancedb.table.Table.search].

@@ -151,7 +150,7 @@ class Table(ABC):
        mode: str = "append",
        on_bad_vectors: str = "error",
        fill_value: float = 0.0,
-    ) -> int:
+    ):
        """Add more data to the [Table](Table).

        Parameters
@@ -167,10 +166,6 @@ class Table(ABC):
        fill_value: float, default 0.
            The value to use when filling vectors. Only used if on_bad_vectors="fill".

-        Returns
-        -------
-        int
-            The number of vectors in the table.
        """
        raise NotImplementedError

@@ -202,6 +197,51 @@ class Table(ABC):
    def _execute_query(self, query: Query) -> pa.Table:
        pass

+    @abstractmethod
+    def delete(self, where: str):
+        """Delete rows from the table.
+
+        This can be used to delete a single row, many rows, all rows, or
+        sometimes no rows (if your predicate matches nothing).
+
+        Parameters
+        ----------
+        where: str
+            The SQL where clause to use when deleting rows. For example, 'x = 2'
+            or 'x IN (1, 2, 3)'. The filter must not be empty, or it will error.
+
+        Examples
+        --------
+        >>> import lancedb
+        >>> import pandas as pd
+        >>> data = pd.DataFrame({"x": [1, 2, 3], "vector": [[1, 2], [3, 4], [5, 6]]})
+        >>> db = lancedb.connect("./.lancedb")
+        >>> table = db.create_table("my_table", data)
+        >>> table.to_pandas()
+           x      vector
+        0  1  [1.0, 2.0]
+        1  2  [3.0, 4.0]
+        2  3  [5.0, 6.0]
+        >>> table.delete("x = 2")
+        >>> table.to_pandas()
+           x      vector
+        0  1  [1.0, 2.0]
+        1  3  [5.0, 6.0]
+
+        If you have a list of values to delete, you can combine them into a
+        stringified list and use the `IN` operator:
+
+        >>> to_remove = [1, 5]
+        >>> to_remove = ", ".join([str(v) for v in to_remove])
+        >>> to_remove
+        '1, 5'
+        >>> table.delete(f"x IN ({to_remove})")
+        >>> table.to_pandas()
+           x      vector
+        0  3  [5.0, 6.0]
+        """
+        raise NotImplementedError
+

 class LanceTable(Table):
    """
@@ -262,7 +302,6 @@ class LanceTable(Table):
               vector    type
        0  [1.1, 0.9]  vector
        >>> table.add([{"vector": [0.5, 0.2], "type": "vector"}])
-        2
        >>> table.version
        2
        >>> table.checkout(1)
@@ -364,7 +403,7 @@ class LanceTable(Table):
        mode: str = "append",
        on_bad_vectors: str = "error",
        fill_value: float = 0.0,
-    ) -> int:
+    ):
        """Add data to the table.

        Parameters
@@ -391,7 +430,6 @@ class LanceTable(Table):
        )
        lance.write_dataset(data, self._dataset_uri, mode=mode)
        self._reset_dataset()
-        return len(self)

    def search(
        self, query: Union[VEC, str], vector_column_name=VECTOR_COLUMN_NAME
@@ -489,38 +527,15 @@ class LanceTable(Table):
    @classmethod
    def open(cls, db, name):
        tbl = cls(db, name)
-        if not os.path.exists(tbl._dataset_uri):
+        fs, path = pa.fs.FileSystem.from_uri(tbl._dataset_uri)
+        file_info = fs.get_file_info(path)
+        if file_info.type != pa.fs.FileType.Directory:
            raise FileNotFoundError(
                f"Table {name} does not exist. Please first call db.create_table({name}, data)"
            )
        return tbl

    def delete(self, where: str):
-        """Delete rows from the table.
-
-        Parameters
-        ----------
-        where: str
-            The SQL where clause to use when deleting rows.
-
-        Examples
-        --------
-        >>> import lancedb
-        >>> import pandas as pd
-        >>> data = pd.DataFrame({"x": [1, 2, 3], "vector": [[1, 2], [3, 4], [5, 6]]})
-        >>> db = lancedb.connect("./.lancedb")
-        >>> table = db.create_table("my_table", data)
-        >>> table.to_pandas()
-           x      vector
-        0  1  [1.0, 2.0]
-        1  2  [3.0, 4.0]
-        2  3  [5.0, 6.0]
-        >>> table.delete("x = 2")
-        >>> table.to_pandas()
-           x      vector
-        0  1  [1.0, 2.0]
-        1  3  [5.0, 6.0]
-        """
        self._dataset.delete(where)

    def _execute_query(self, query: Query) -> pa.Table:
--- a/python/lancedb/util.py
+++ b/python/lancedb/util.py
@@ -11,8 +11,13 @@
 #  See the License for the specific language governing permissions and
 #  limitations under the License.

+import os
+from typing import Tuple
 from urllib.parse import urlparse

+import pyarrow as pa
+import pyarrow.fs as pa_fs
+

 def get_uri_scheme(uri: str) -> str:
    """
@@ -59,3 +64,14 @@ def get_uri_location(uri: str) -> str:
        return parsed.path
    else:
        return parsed.netloc + parsed.path
+
+
+def fs_from_uri(uri: str) -> Tuple[pa_fs.FileSystem, str]:
+    """
+    Get a PyArrow FileSystem from a URI, handling extra environment variables.
+    """
+    if get_uri_scheme(uri) == "s3":
+        if os.environ["AWS_ENDPOINT"]:
+            uri += "?endpoint_override=" + os.environ["AWS_ENDPOINT"]
+
+    return pa_fs.FileSystem.from_uri(uri)
--- a/python/pyproject.toml
+++ b/python/pyproject.toml
@@ -1,7 +1,7 @@
 [project]
 name = "lancedb"
-version = "0.1.11"
-dependencies = ["pylance~=0.5.8", "ratelimiter", "retry", "tqdm", "aiohttp", "pydantic>=2", "attr"]
+version = "0.1.13"
+dependencies = ["pylance~=0.5.8", "ratelimiter", "retry", "tqdm", "aiohttp", "pydantic", "attr", "semver"]
 description = "lancedb"
 authors = [
    { name = "LanceDB Devs", email = "dev@lancedb.com" },
@@ -52,3 +52,6 @@ requires = [
  "wheel",
 ]
 build-backend = "setuptools.build_meta"
+
+[tool.isort]
+profile = "black"
--- a/python/tests/test_pydantic.py
+++ b/python/tests/test_pydantic.py
@@ -20,7 +20,7 @@ import pyarrow as pa
 import pydantic
 import pytest

-from lancedb.pydantic import pydantic_to_schema, vector
+from lancedb.pydantic import PYDANTIC_VERSION, pydantic_to_schema, vector


@pytest.mark.skipif(
@@ -111,10 +111,16 @@ def test_fixed_size_list_field():
        li: List[int]

    data = TestModel(vec=list(range(16)), li=[1, 2, 3])
-    assert json.loads(data.model_dump_json()) == {
-        "vec": list(range(16)),
-        "li": [1, 2, 3],
-    }
+    if PYDANTIC_VERSION >= (2,):
+        assert json.loads(data.model_dump_json()) == {
+            "vec": list(range(16)),
+            "li": [1, 2, 3],
+        }
+    else:
+        assert data.dict() == {
+            "vec": list(range(16)),
+            "li": [1, 2, 3],
+        }

    schema = pydantic_to_schema(TestModel)
    assert schema == pa.schema(
@@ -124,7 +130,11 @@ def test_fixed_size_list_field():
        ]
    )

-    json_schema = TestModel.model_json_schema()
+    if PYDANTIC_VERSION >= (2,):
+        json_schema = TestModel.model_json_schema()
+    else:
+        json_schema = TestModel.schema()
+
    assert json_schema == {
        "properties": {
            "vec": {
--- a/python/tests/test_table.py
+++ b/python/tests/test_table.py
@@ -139,8 +139,8 @@ def _add(table, schema):
    # table = LanceTable(db, "test")
    assert len(table) == 2

-    count = table.add([{"vector": [6.3, 100.5], "item": "new", "price": 30.0}])
-    assert count == 3
+    table.add([{"vector": [6.3, 100.5], "item": "new", "price": 30.0}])
+    assert len(table) == 3

    expected = pa.Table.from_arrays(
        [
--- a/rust/ffi/node/Cargo.toml
+++ b/rust/ffi/node/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "vectordb-node"
-version = "0.1.13"
+version = "0.1.15"
 description = "Serverless, low-latency vector database for AI applications"
 license = "Apache-2.0"
 edition = "2018"
--- a/rust/ffi/node/src/lib.rs
+++ b/rust/ffi/node/src/lib.rs
@@ -21,7 +21,7 @@ use arrow_array::{Float32Array, RecordBatchIterator};
 use arrow_ipc::writer::FileWriter;
 use async_trait::async_trait;
 use futures::{TryFutureExt, TryStreamExt};
-use lance::dataset::{ReadParams, WriteMode, WriteParams};
+use lance::dataset::{WriteMode, WriteParams};
 use lance::index::vector::MetricType;
 use lance::io::object_store::ObjectStoreParams;
 use neon::prelude::*;
@@ -33,7 +33,7 @@ use tokio::runtime::Runtime;

 use vectordb::database::Database;
 use vectordb::error::Error;
-use vectordb::table::{OpenTableParams, Table};
+use vectordb::table::{ReadParams, Table};

 use crate::arrow::arrow_buffer_to_record_batch;

@@ -177,7 +177,7 @@ fn database_open_table(mut cx: FunctionContext) -> JsResult<JsPromise> {
        Err(err) => return err,
    };

-    let param = ReadParams {
+    let params = ReadParams {
        store_options: Some(ObjectStoreParams {
            aws_credentials: aws_creds,
            ..ObjectStoreParams::default()
@@ -191,14 +191,7 @@ fn database_open_table(mut cx: FunctionContext) -> JsResult<JsPromise> {

    let (deferred, promise) = cx.promise();
    rt.spawn(async move {
-        let table_rst = database
-            .open_table_with_params(
-                &table_name,
-                OpenTableParams {
-                    open_table_params: param,
-                },
-            )
-            .await;
+        let table_rst = database.open_table_with_params(&table_name, &params).await;

        deferred.settle_with(&channel, move |mut cx| {
            let table = Arc::new(Mutex::new(
--- a/rust/vectordb/Cargo.toml
+++ b/rust/vectordb/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "vectordb"
-version = "0.1.13"
+version = "0.1.15"
 edition = "2021"
 description = "Serverless, low-latency vector database for AI applications"
 license = "Apache-2.0"
--- a/rust/vectordb/src/database.rs
+++ b/rust/vectordb/src/database.rs
@@ -20,8 +20,10 @@ use lance::dataset::WriteParams;
 use lance::io::object_store::ObjectStore;
 use snafu::prelude::*;

-use crate::error::{CreateDirSnafu, Result};
-use crate::table::{OpenTableParams, Table};
+use crate::error::{CreateDirSnafu, InvalidTableNameSnafu, Result};
+use crate::table::{ReadParams, Table};
+
+pub const LANCE_FILE_EXTENSION: &str = "lance";

 pub struct Database {
    object_store: ObjectStore,
@@ -59,7 +61,7 @@ impl Database {
    fn try_create_dir(path: &str) -> core::result::Result<(), std::io::Error> {
        let path = Path::new(path);
        if !path.try_exists()? {
-            create_dir_all(&path)?;
+            create_dir_all(path)?;
        }
        Ok(())
    }
@@ -75,20 +77,15 @@ impl Database {
            .read_dir(self.base_path.clone())
            .await?
            .iter()
-            .map(|fname| Path::new(fname))
+            .map(Path::new)
            .filter(|path| {
                let is_lance = path
                    .extension()
-                    .map(|e| e.to_str().map(|e| e == LANCE_EXTENSION))
-                    .flatten();
+                    .and_then(|e| e.to_str())
+                    .map(|e| e == LANCE_EXTENSION);
                is_lance.unwrap_or(false)
            })
-            .map(|p| {
-                p.file_stem()
-                    .map(|s| s.to_str().map(|s| String::from(s)))
-                    .flatten()
-            })
-            .flatten()
+            .filter_map(|p| p.file_stem().and_then(|s| s.to_str().map(String::from)))
            .collect();
        Ok(f)
    }
@@ -105,7 +102,8 @@ impl Database {
        batches: impl RecordBatchReader + Send + 'static,
        params: Option<WriteParams>,
    ) -> Result<Table> {
-        Table::create(&self.uri, name, batches, params).await
+        let table_uri = self.table_uri(name)?;
+        Table::create(&table_uri, name, batches, params).await
    }

    /// Open a table in the database.
@@ -117,7 +115,7 @@ impl Database {
    ///
    /// * A [Table] object.
    pub async fn open_table(&self, name: &str) -> Result<Table> {
-        self.open_table_with_params(name, OpenTableParams::default())
+        self.open_table_with_params(name, &ReadParams::default())
            .await
    }

@@ -130,12 +128,9 @@ impl Database {
    /// # Returns
    ///
    /// * A [Table] object.
-    pub async fn open_table_with_params(
-        &self,
-        name: &str,
-        params: OpenTableParams,
-    ) -> Result<Table> {
-        Table::open_with_params(&self.uri, name, params).await
+    pub async fn open_table_with_params(&self, name: &str, params: &ReadParams) -> Result<Table> {
+        let table_uri = self.table_uri(name)?;
+        Table::open_with_params(&table_uri, name, params).await
    }

    /// Drop a table in the database.
@@ -148,6 +143,18 @@ impl Database {
        self.object_store.remove_dir_all(full_path).await?;
        Ok(())
    }
+
+    /// Get the URI of a table in the database.
+    fn table_uri(&self, name: &str) -> Result<String> {
+        let path = Path::new(&self.uri);
+        let table_uri = path.join(format!("{}.{}", name, LANCE_FILE_EXTENSION));
+
+        let uri = table_uri
+            .as_path()
+            .to_str()
+            .context(InvalidTableNameSnafu { name })?;
+        Ok(uri.to_string())
+    }
 }

 #[cfg(test)]
--- a/rust/vectordb/src/index/vector.rs
+++ b/rust/vectordb/src/index/vector.rs
@@ -35,6 +35,12 @@ pub struct IvfPQIndexBuilder {

 impl IvfPQIndexBuilder {
    pub fn new() -> IvfPQIndexBuilder {
+        Default::default()
+    }
+}
+
+impl Default for IvfPQIndexBuilder {
+    fn default() -> Self {
        IvfPQIndexBuilder {
            column: None,
            index_name: None,
--- a/rust/vectordb/src/table.rs
+++ b/rust/vectordb/src/table.rs
@@ -12,22 +12,22 @@
 // See the License for the specific language governing permissions and
 // limitations under the License.

-use std::path::Path;
 use std::sync::Arc;

 use arrow_array::{Float32Array, RecordBatchReader};
 use arrow_schema::SchemaRef;
-use lance::dataset::{Dataset, ReadParams, WriteParams};
+use lance::dataset::{Dataset, WriteParams};
 use lance::index::IndexType;
-use snafu::prelude::*;
+use std::path::Path;

-use crate::error::{Error, InvalidTableNameSnafu, Result};
+use crate::error::{Error, Result};
 use crate::index::vector::VectorIndexBuilder;
 use crate::query::Query;
 use crate::WriteMode;

+pub use lance::dataset::ReadParams;
+
 pub const VECTOR_COLUMN_NAME: &str = "vector";
-pub const LANCE_FILE_EXTENSION: &str = "lance";

 /// A table in a LanceDB database.
 #[derive(Debug, Clone)]
@@ -43,24 +43,25 @@ impl std::fmt::Display for Table {
    }
 }

-#[derive(Default)]
-pub struct OpenTableParams {
-    pub open_table_params: ReadParams,
-}
-
 impl Table {
    /// Opens an existing Table
    ///
    /// # Arguments
    ///
-    /// * `base_path` - The base path where the table is located
-    /// * `name` The Table name
+    /// * `uri` - The uri to a [Table]
+    /// * `name` - The table name
    ///
    /// # Returns
    ///
    /// * A [Table] object.
-    pub async fn open(base_uri: &str, name: &str) -> Result<Self> {
-        Self::open_with_params(base_uri, name, OpenTableParams::default()).await
+    pub async fn open(uri: &str) -> Result<Self> {
+        let name = Self::get_table_name(uri)?;
+        Self::open_with_params(uri, &name, &ReadParams::default()).await
+    }
+
+    /// Open an Table with a given name.
+    pub async fn open_with_name(uri: &str, name: &str) -> Result<Self> {
+        Self::open_with_params(uri, name, &ReadParams::default()).await
    }

    /// Opens an existing Table
@@ -69,25 +70,13 @@ impl Table {
    ///
    /// * `base_path` - The base path where the table is located
    /// * `name` The Table name
-    /// * `params` The [OpenTableParams] to use when opening the table
+    /// * `params` The [ReadParams] to use when opening the table
    ///
    /// # Returns
    ///
    /// * A [Table] object.
-    pub async fn open_with_params(
-        base_uri: &str,
-        name: &str,
-        params: OpenTableParams,
-    ) -> Result<Self> {
-        let path = Path::new(base_uri);
-
-        let table_uri = path.join(format!("{}.{}", name, LANCE_FILE_EXTENSION));
-        let uri = table_uri
-            .as_path()
-            .to_str()
-            .context(InvalidTableNameSnafu { name })?;
-
-        let dataset = Dataset::open_with_params(uri, &params.open_table_params)
+    pub async fn open_with_params(uri: &str, name: &str, params: &ReadParams) -> Result<Self> {
+        let dataset = Dataset::open_with_params(uri, params)
            .await
            .map_err(|e| match e {
                lance::Error::DatasetNotFound { .. } => Error::TableNotFound {
@@ -104,31 +93,73 @@ impl Table {
        })
    }

+    /// Checkout a specific version of this [`Table`]
+    ///
+    pub async fn checkout(uri: &str, version: u64) -> Result<Self> {
+        let name = Self::get_table_name(uri)?;
+        Self::checkout_with_params(uri, &name, version, &ReadParams::default()).await
+    }
+
+    pub async fn checkout_with_name(uri: &str, name: &str, version: u64) -> Result<Self> {
+        Self::checkout_with_params(uri, name, version, &ReadParams::default()).await
+    }
+
+    pub async fn checkout_with_params(
+        uri: &str,
+        name: &str,
+        version: u64,
+        params: &ReadParams,
+    ) -> Result<Self> {
+        let dataset = Dataset::checkout_with_params(uri, version, params)
+            .await
+            .map_err(|e| match e {
+                lance::Error::DatasetNotFound { .. } => Error::TableNotFound {
+                    name: name.to_string(),
+                },
+                e => Error::Lance {
+                    message: e.to_string(),
+                },
+            })?;
+        Ok(Table {
+            name: name.to_string(),
+            uri: uri.to_string(),
+            dataset: Arc::new(dataset),
+        })
+    }
+
+    fn get_table_name(uri: &str) -> Result<String> {
+        let path = Path::new(uri);
+        let name = path
+            .file_stem()
+            .ok_or(Error::TableNotFound {
+                name: uri.to_string(),
+            })?
+            .to_str()
+            .ok_or(Error::InvalidTableName {
+                name: uri.to_string(),
+            })?;
+        Ok(name.to_string())
+    }
+
    /// Creates a new Table
    ///
    /// # Arguments
    ///
-    /// * `base_path` - The base path where the table is located
+    /// * `uri` - The URI to the table.
    /// * `name` The Table name
-    /// * `batches` RecordBatch to be saved in the database
+    /// * `batches` RecordBatch to be saved in the database.
+    /// * `params` - Write parameters.
    ///
    /// # Returns
    ///
    /// * A [Table] object.
    pub async fn create(
-        base_uri: &str,
+        uri: &str,
        name: &str,
        batches: impl RecordBatchReader + Send + 'static,
        params: Option<WriteParams>,
    ) -> Result<Self> {
-        let base_path = Path::new(base_uri);
-        let table_uri = base_path.join(format!("{}.{}", name, LANCE_FILE_EXTENSION));
-        let uri = table_uri
-            .as_path()
-            .to_str()
-            .context(InvalidTableNameSnafu { name })?
-            .to_string();
-        let dataset = Dataset::write(batches, &uri, params)
+        let dataset = Dataset::write(batches, uri, params)
            .await
            .map_err(|e| match e {
                lance::Error::DatasetAlreadyExists { .. } => Error::TableAlreadyExists {
@@ -140,7 +171,7 @@ impl Table {
            })?;
        Ok(Table {
            name: name.to_string(),
-            uri,
+            uri: uri.to_string(),
            dataset: Arc::new(dataset),
        })
    }
@@ -264,14 +295,13 @@ mod tests {
    async fn test_open() {
        let tmp_dir = tempdir().unwrap();
        let dataset_path = tmp_dir.path().join("test.lance");
-        let uri = tmp_dir.path().to_str().unwrap();

        let batches = make_test_batches();
        Dataset::write(batches, dataset_path.to_str().unwrap(), None)
            .await
            .unwrap();

-        let table = Table::open(uri, "test").await.unwrap();
+        let table = Table::open(dataset_path.to_str().unwrap()).await.unwrap();

        assert_eq!(table.name, "test")
    }
@@ -280,7 +310,7 @@ mod tests {
    async fn test_open_not_found() {
        let tmp_dir = tempdir().unwrap();
        let uri = tmp_dir.path().to_str().unwrap();
-        let table = Table::open(uri, "test").await;
+        let table = Table::open(uri).await;
        assert!(matches!(table.unwrap_err(), Error::TableNotFound { .. }));
    }

@@ -371,14 +401,14 @@ mod tests {
    async fn test_search() {
        let tmp_dir = tempdir().unwrap();
        let dataset_path = tmp_dir.path().join("test.lance");
-        let uri = tmp_dir.path().to_str().unwrap();
+        let uri = dataset_path.to_str().unwrap();

        let batches = make_test_batches();
        Dataset::write(batches, dataset_path.to_str().unwrap(), None)
            .await
            .unwrap();

-        let table = Table::open(uri, "test").await.unwrap();
+        let table = Table::open(uri).await.unwrap();

        let vector = Float32Array::from_iter_values([0.1, 0.2]);
        let query = table.search(vector.clone());
@@ -410,7 +440,7 @@ mod tests {
    async fn test_open_table_options() {
        let tmp_dir = tempdir().unwrap();
        let dataset_path = tmp_dir.path().join("test.lance");
-        let uri = tmp_dir.path().to_str().unwrap();
+        let uri = dataset_path.to_str().unwrap();

        let batches = make_test_batches();
        Dataset::write(batches, dataset_path.to_str().unwrap(), None)
@@ -421,15 +451,12 @@ mod tests {

        let mut object_store_params = ObjectStoreParams::default();
        object_store_params.object_store_wrapper = Some(wrapper.clone());
-        let param = OpenTableParams {
-            open_table_params: ReadParams {
-                store_options: Some(object_store_params),
-                ..ReadParams::default()
-            },
+        let param = ReadParams {
+            store_options: Some(object_store_params),
+            ..Default::default()
        };
-
        assert!(!wrapper.called());
-        let _ = Table::open_with_params(uri, "test", param).await.unwrap();
+        let _ = Table::open_with_params(uri, "test", &param).await.unwrap();
        assert!(wrapper.called());
    }
Author	SHA1	Message	Date
Lance Release	f4ce86e12c	[python] Bump version: 0.1.12 → 0.1.13	2023-07-19 03:09:50 +00:00
Lance Release	0664eaec82	Bump version: 0.1.14 → 0.1.15	2023-07-19 02:54:10 +00:00
Lei Xu	63acdc2069	[Python] Support pydantic v1 as well (#337 ) Support both Pydantic v1 and v2 (breaking changes)	2023-07-18 19:53:09 -07:00
Rob Meng	a636bb1075	add support for host override (#335 )	2023-07-18 21:21:39 -04:00
Lance Release	5e3167da83	[python] Bump version: 0.1.11 → 0.1.12	2023-07-19 01:18:28 +00:00
Lei Xu	f09db4a6d6	[Python] Do not return Table count for every add operation (#328 ) `Table::count()` will be linearly slower with more fragments ingested.	2023-07-18 17:11:17 -07:00
Lei Xu	1d343edbd4	[Node] implement remote db.TableNames (#334 )	2023-07-18 16:56:47 -07:00
Lei Xu	980f910f50	[Node] initial support of nodejs remote sdk (#333 )	2023-07-18 16:15:27 -07:00
Will Jones	fb97b03a51	feat: pass AWS_ENDPOINT environment variable down (#330 ) Tested locally against minio.	2023-07-18 15:07:26 -07:00
Lei Xu	141b6647a8	[Python] Fix bumpversion.cfg (#327 )	2023-07-18 09:18:14 -07:00
gsilvestrin	b45ac4608f	feat(node): Explicitly set registry url when publishing package (#324 )	2023-07-18 08:55:56 -07:00
Lei Xu	a86bc05131	[Bug] Fix dataset path check in Table::open (#326 ) Fixed a bug that prevents to open remote tables.	2023-07-18 08:45:10 -07:00
Will Jones	3537afb2c3	docs: show how to delete rows in user guide (#309 ) Closes #265	2023-07-18 08:19:48 -07:00
Lei Xu	23f5dddc7c	[Rust] Checkout a version of dataset. (#321 ) * `Table::open()` from absolute path, and gives the responsibility of organizing metadata out of Table object * Fix Clippy warnings * Add `Table::checkout(version)` API	2023-07-17 17:29:58 -07:00
gsilvestrin	9748406cba	Updating package-lock.json (#322 )	2023-07-17 16:48:22 -07:00
gsilvestrin	6271949d38	feat(node): Update package-lock.json on each release (#302 )	2023-07-17 16:33:43 -07:00
Lance Release	131ad09ab3	Bump version: 0.1.13 → 0.1.14	2023-07-17 20:06:58 +00:00