rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-01-05 11:22:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
812 Commits 115 Branches 397 Tags
4605359d3ba970cb4d8719dcbc1ca8d789a02bf0
Commit Graph

97 Commits

Author SHA1 Message Date
Will Jones
3aa0c40168 feat(node): add read_consistency_interval to Node and Rust (#1002) 
This PR adds the same consistency semantics as was added in #828. It
*does not* add the same lazy-loading of tables, since that breaks some
existing tests.

This closes #998.

---------

Co-authored-by: Weston Pace <weston.pace@gmail.com>
 2024-02-22 15:04:30 -08:00
Weston Pace
cc5f2136a6 feat: make it easier to create empty tables (#942) 
This PR also reworks the table creation utilities significantly so that
they are more consistent, built on top of each other, and thoroughly
documented.
 2024-02-13 10:51:18 -08:00
Will Jones
8104c5c18e fix: wrap in BigInt to avoid upstream bug (#962) 
Closes #960
 2024-02-13 08:13:56 -08:00
Weston Pace
a9727eb318 feat: add support for filter during merge insert when matched (#948) 
Closes #940
 2024-02-09 10:26:14 -08:00
Weston Pace
d2e71c8b08 feat: add a filterable count_rows to all the lancedb APIs (#913) 
A `count_rows` method that takes a filter was recently added to
`LanceTable`. This PR adds it everywhere else except `RemoteTable` (that
will come soon).
 2024-02-08 09:40:29 -08:00
Weston Pace
7f8637a0b4 feat: add merge_insert to the node and rust APIs (#915) 2024-02-02 13:16:51 -08:00
Lei Xu
5f59e51583 fix(node): pass AWS credentials to db level operations (#908) 
Passed the following tests

```ts
const keyId = process.env.AWS_ACCESS_KEY_ID;
const secretKey = process.env.AWS_SECRET_ACCESS_KEY;
const sessionToken = process.env.AWS_SESSION_TOKEN;
const region = process.env.AWS_REGION;

const db = await lancedb.connect({
  uri: "s3://bucket/path",
  awsCredentials: {
    accessKeyId: keyId,
    secretKey: secretKey,
    sessionToken: sessionToken,
  },
  awsRegion: region,
} as lancedb.ConnectionOptions);

  console.log(await db.createTable("test", [{ vector: [1, 2, 3] }]));
  console.log(await db.tableNames());
  console.log(await db.dropTable("test"))
```
 2024-01-31 12:05:01 -08:00
Lei Xu
22b9eceb12 chore: convert all js doc test to use snippet. (#881) 2024-01-28 11:39:25 -08:00
Lei Xu
e01ef63488 chore(rust): simplified version of optimize (#869) 
Consolidate various optimize() into one method, similar to postgres
VACCUM in the process of preparing Rust API for public use
 2024-01-26 11:36:04 -08:00
Lei Xu
ccfd043939 feat: change create table to accept Arrow table (#845) 2024-01-23 13:25:15 -08:00
Bert
66eaa2a00e allow passing api key as env var (#841) 
Allow passing API key as env var:
```shell
export LANCEDB_API_KEY=sh_123...
```

with this set, apiKey argument can omitted from `connect`
```js
    const db = await vectordb.connect({
        uri: "db://test-proj-01-ae8343",
        region: "us-east-1",
  })
```
```py
    db = lancedb.connect(
        uri="db://test-proj-01-ae8343",
        region="us-east-1",
    )
```
 2024-01-22 16:18:28 -05:00
Lei Xu
5f14a411af feat(js): add helper function to create Arrow Table with schema (#838) 
Support to make Apache Arrow Table from an array of javascript Records,
with optionally provided Schema.
 2024-01-22 11:49:44 -08:00
Chang She
bea3cef627 chore(js): remove errant console.log (#834) 2024-01-22 11:44:38 -08:00
Chang She
3ba1618be9 Merge branch 'tecmie/embeddings-openai' of github.com:tecmie/lancedb into tecmie-tecmie/embeddings-openai 2024-01-19 16:45:41 -08:00
Lei Xu
9a9fc77a95 doc: improve docs for nodejs connect functions (#833) 
* improve the docstring for NodeJS connect functions and
`ConnectOptions` parameters.
* Simplify `npm run build` steps.
 2024-01-19 16:07:53 -08:00
Andrew Miracle
44eba363b5 eslint fix 2024-01-13 09:15:01 +01:00
Andrew Miracle
f7f9beaf31 rebase from lancedb/main 2024-01-12 10:17:30 +01:00
Chang She
3f66be666d feat(node): align incoming data to table schema (#802) 2024-01-12 09:45:40 +01:00
Chang She
a649b3b1e4 feat(node): support table.schema for LocalTable (#789) 
Close #773 

we pass an empty table over IPC so we don't need to manually deal with
serde. Then we just return the schema attribute from the empty table.

---------

Co-authored-by: albertlockett <albert.lockett@gmail.com>
 2024-01-12 09:45:36 +01:00
Chang She
4417f7c5a7 feat(js): support list of string input (#755) 
Add support for adding lists of string input (e.g., list of categorical
labels)

Follow-up items: #757 #758
 2024-01-12 09:45:36 +01:00
Aidan
3d8b2f5531 fix: createIndex index cache size (#741) 2024-01-12 09:45:29 +01:00
Chang She
81af350d85 feat(node): align incoming data to table schema (#802) 2024-01-10 16:44:00 -08:00
Andrew Miracle
821cf0e434 eslint fix 2024-01-09 16:27:22 +01:00
Chang She
ac40d4b235 feat(node): support table.schema for LocalTable (#789) 
Close #773 

we pass an empty table over IPC so we don't need to manually deal with
serde. Then we just return the schema attribute from the empty table.

---------

Co-authored-by: albertlockett <albert.lockett@gmail.com>
 2024-01-08 21:12:48 -08:00
Chang She
684eb8b087 feat(js): support list of string input (#755) 
Add support for adding lists of string input (e.g., list of categorical
labels)

Follow-up items: #757 #758
 2024-01-02 20:55:33 -08:00
Aidan
446f837335 fix: createIndex index cache size (#741) 2023-12-27 09:25:13 -08:00
Andrew Miracle
ee1d0b596f remove console logs 2023-12-25 21:51:02 +00:00
Andrew Miracle
38a4524893 add support for openai SDK version ^4.24.1 2023-12-25 20:29:54 +00:00
Bert
756188358c docs: fix JS api docs for update method (#738) 2023-12-21 13:48:00 -05:00
Weston Pace
dc5126d8d1 feat: add the ability to create scalar indices (#679) 
This is a pretty direct binding to the underlying lance capability
 2023-12-21 09:50:10 -08:00
Aidan
50c20af060 feat: node list tables pagination (#733) 2023-12-21 11:37:19 -05:00
Aidan
fff8e399a3 feat: Node create index API (#720) 2023-12-20 15:22:35 -05:00
Aidan
73e4015797 feat: Node Schema API (#717) 2023-12-20 12:16:40 -05:00
Bert
57207eff4a implement update for remote clients (#706) 2023-12-15 09:06:40 -05:00
Chang She
098e397cf0 feat: LocalTable for vectordb now supports filters without vector search (#693) 
Note this currently the filter/where is only implemented for LocalTable
so that it requires an explicit cast to "enable" (see new unit test).
The alternative is to add it to the Table interface, but since it's not
available on RemoteTable this may cause some user experience issues.
 2023-12-13 22:59:01 -08:00
Bert
63ee8fa6a1 Update in Node & Rust (#696) 
Co-authored-by: Will Jones <willjones127@gmail.com>
 2023-12-13 14:53:06 -05:00
Rob Meng
94c8c50f96 fix: fix passing prefilter flag to remote client (#677) 
was passing this at the wrong position
 2023-12-04 12:01:16 -05:00
Rob Meng
72765d8e1a feat: enable prefilter in node js (#675) 
enable prefiltering in node js, both native and remote
 2023-12-01 16:49:10 -05:00
Will Jones
a57aa4b142 chore: upgrade lance to v0.8.17 (#656) 
Readying for the next Lance release.
 2023-11-18 15:57:23 -08:00
Bert
797514bcbf fix: node remote implement table.countRows (#648) 2023-11-13 17:43:20 -05:00
Bert
479f471c14 fix: node send db header for GET requests (#646) 2023-11-11 16:33:25 -05:00
Bert
567734dd6e fix: node remote connection handles non http errors (#624) 
https://github.com/lancedb/lancedb/issues/623

Fixes issue trying to print response status when using remote client. If
the error is not an HTTP error (e.g. dns/network failure), there won't
be a response.
 2023-11-03 10:24:56 -04:00
Bert
5ca98c326f feat: added dataset stats api to node (#604) 2023-10-26 17:00:48 -04:00
Rob Meng
ce19fedb08 feat: include manifest files in mirrow store (#589) 2023-10-21 12:21:41 -04:00
Rob Meng
345c136cfb implement remote api calls for table mutation (#567) 
Add more APIs to remote table for Node SDK
* `add` rows
* `overwrite` table with rows
* `create` table

This has been tested against dev stack
 2023-10-16 11:07:58 -04:00
Lei Xu
fe64fc4671 feat(python,js): deletion operation on remote tables (#568) 2023-10-14 15:47:19 -07:00
Will Jones
db7bdefe77 feat: cleanup and compaction (#518) 
#488
 2023-10-11 12:49:12 -07:00
Rob Meng
1db66c6980 implement mirroring object store (#537) 
This PR implements a mirroring object store and allows and table to be
mirrored to a local path when param `mirroredStore` is set in the url
 2023-10-04 21:23:34 -04:00
Rob Meng
dbf37a0434 fix: upgrade lance to 0.7.5 and add tests for searching empty dataset (#505) 
This PR upgrade lance to `0.7.5`, which include fixes for searching an
empty dataset.

This PR also adds two tests in node SDK to make sure searching empty
dataset do no throw

Co-authored-by: rmeng <rob@lancedb.com>
 2023-09-18 22:12:11 -07:00
Rob Meng
731f86e44c add health check to wait for all service ready before next step (#501) 
aws integration tests are flaky because we didn't wait for the services
to become healthy. (we only waited for the localstack service, this PR
adds wait for sub services)
 2023-09-18 15:17:45 -04:00