Commit Graph

51 Commits

Author SHA1 Message Date
Rob Meng f3de3d990d chore: upgrade to lance 0.10.1 (#1034)
upgrade to lance 0.10.1 and update doc string to reflect dynamic
projection options
2024-04-05 16:31:36 -07:00
Will Jones 464a36ad38 feat: {add|alter|drop}_columns APIs (#1015)
Initial work for #959. This exposes the basic functionality for each in
all of the APIs. Will add user guide documentation in a later PR.
2024-04-05 16:30:47 -07:00
Weston Pace 9241f47f0e feat: make it easier to create empty tables (#942)
This PR also reworks the table creation utilities significantly so that
they are more consistent, built on top of each other, and thoroughly
documented.
2024-04-05 16:30:30 -07:00
Weston Pace 41ccb48160 feat: add support for filter during merge insert when matched (#948)
Closes #940
2024-04-05 16:29:58 -07:00
Weston Pace 138fc3f66b feat: add a filterable count_rows to all the lancedb APIs (#913)
A `count_rows` method that takes a filter was recently added to
`LanceTable`. This PR adds it everywhere else except `RemoteTable` (that
will come soon).
2024-04-05 16:29:58 -07:00
Weston Pace 18f7bad3dd feat: add merge_insert to the node and rust APIs (#915) 2024-04-05 16:29:05 -07:00
Lei Xu a192c1a9b1 chore(rust): simplified version of optimize (#869)
Consolidate various optimize() into one method, similar to postgres
VACCUM in the process of preparing Rust API for public use
2024-04-05 16:28:18 -07:00
Lei Xu 65c1d8bc4c feat: change create table to accept Arrow table (#845) 2024-04-05 16:27:50 -07:00
Lei Xu d8befeeea2 feat(js): add helper function to create Arrow Table with schema (#838)
Support to make Apache Arrow Table from an array of javascript Records,
with optionally provided Schema.
2024-04-05 16:27:42 -07:00
Andrew Miracle 8daed93a91 eslint fix 2024-04-05 16:25:52 -07:00
Andrew Miracle fa13fb9392 rebase from lancedb/main 2024-04-05 16:25:14 -07:00
Chang She 118a11c9b3 feat(node): align incoming data to table schema (#802) 2024-04-05 16:25:14 -07:00
Chang She a758876a65 feat(node): support table.schema for LocalTable (#789)
Close #773 

we pass an empty table over IPC so we don't need to manually deal with
serde. Then we just return the schema attribute from the empty table.

---------

Co-authored-by: albertlockett <albert.lockett@gmail.com>
2024-04-05 16:25:14 -07:00
Chang She 81487f10fe feat(js): support list of string input (#755)
Add support for adding lists of string input (e.g., list of categorical
labels)

Follow-up items: #757 #758
2024-04-05 16:25:02 -07:00
Andrew Miracle 5948f11641 eslint fix 2024-04-05 16:25:02 -07:00
Andrew Miracle 453bf113ae add support for openai SDK version ^4.24.1 2024-04-05 16:25:02 -07:00
Weston Pace 94e81ff84b feat: add the ability to create scalar indices (#679)
This is a pretty direct binding to the underlying lance capability
2024-04-05 16:24:47 -07:00
Chang She b02370cacd feat: LocalTable for vectordb now supports filters without vector search (#693)
Note this currently the filter/where is only implemented for LocalTable
so that it requires an explicit cast to "enable" (see new unit test).
The alternative is to add it to the Table interface, but since it's not
available on RemoteTable this may cause some user experience issues.
2024-04-05 16:24:15 -07:00
Bert e479acc1bd Update in Node & Rust (#696)
Co-authored-by: Will Jones <willjones127@gmail.com>
2024-04-05 16:24:15 -07:00
Rob Meng 59c25574f0 feat: enable prefilter in node js (#675)
enable prefiltering in node js, both native and remote
2024-04-05 16:23:49 -07:00
Will Jones 6d76fe80b8 chore: upgrade lance to v0.8.17 (#656)
Readying for the next Lance release.
2024-04-05 16:23:49 -07:00
Bert 20ab85171b fix: node remote connection handles non http errors (#624)
https://github.com/lancedb/lancedb/issues/623

Fixes issue trying to print response status when using remote client. If
the error is not an HTTP error (e.g. dns/network failure), there won't
be a response.
2024-04-05 16:23:14 -07:00
Bert 4d086d63eb feat: added dataset stats api to node (#604) 2024-04-05 16:22:59 -07:00
Will Jones c07207c661 feat: cleanup and compaction (#518)
#488
2024-04-05 16:22:59 -07:00
Rob Meng dbf37a0434 fix: upgrade lance to 0.7.5 and add tests for searching empty dataset (#505)
This PR upgrade lance to `0.7.5`, which include fixes for searching an
empty dataset.

This PR also adds two tests in node SDK to make sure searching empty
dataset do no throw

Co-authored-by: rmeng <rob@lancedb.com>
2023-09-18 22:12:11 -07:00
gsilvestrin 2737315cb2 feat(node): Create empty tables / Arrow Tables (#399)
- Supports creating an empty table as long as an Arrow Schema is provided
- Supports creating a table from an Arrow Table (can be passed as data)
- Simplified some Arrow code in the TS/FFI side
- removed createTableArrow method, it was never documented / tested.
2023-08-22 10:57:45 -07:00
Will Jones 722462c38b chore: upgrade Lance and rename score to _distance (#398)
BREAKING CHANGE: The `score` column has been renamed to `_distance` to
more accurately describe the semantics (smaller means closer / better).

---------

Co-authored-by: Lei Xu <lei@lancedb.com>
2023-08-11 21:42:33 -07:00
gsilvestrin bcd7f66dc7 fix(node): Handle overflows in the node bridge (#372)
- Fixes many numeric conversions that results in hard to reproduce issues
- JsObjectExt extends JsObject with safe methods to extract numericvalues
2023-07-28 13:15:21 -07:00
gsilvestrin 73cc12ecc5 fix(node): Relax EmbeddingFunction type guard (#370) 2023-07-27 12:51:59 -07:00
gsilvestrin 6036cf48a7 fix(node) Replace panic errors with friendlier ones (#366)
- Implement Result/Error in the node FFI
- Implement a trait (ResultExt) to make error handling less verbose
- Refactor some parts of the code that touch arrow into arrow.rs
2023-07-26 13:44:58 -07:00
gsilvestrin 80a32be121 bugfix(node): make WriteMode optional when specifying embeddings (#336) 2023-07-24 11:26:43 -07:00
Lei Xu 980f910f50 [Node] initial support of nodejs remote sdk (#333) 2023-07-18 16:15:27 -07:00
gsilvestrin 826dc90151 feat(node): add option object to connect method (#286) 2023-07-13 11:03:48 -07:00
Lei Xu fc725c99f0 [Node] Create Table with WriteMode (#246)
Support `createTable(name, data, mode?)`  to be consistent with Python.

Closes #242
2023-07-03 17:04:21 -07:00
Lei Xu c68c236f17 [Js] Create index with replace flag (#229) 2023-06-26 18:38:20 -07:00
Lei Xu c14ad91df0 [Node] drop table api (#221)
Provide `drop_table` in rust and node. Closes #86
2023-06-23 19:58:37 -07:00
Will Jones ad48242ffb feat: support for deletion (#219)
Also upgrades Arrow and Lance.
2023-06-23 18:09:07 -07:00
gsilvestrin 78de8f5782 feat(node): add Table.countRows() (#185) 2023-06-15 14:35:54 -07:00
gsilvestrin f65d85efcc feat(node): add where method to query builder (#183)
Closes #181
2023-06-14 10:54:43 -07:00
gsilvestrin f37994b72a [nodejs] deprecated created_index in favor of createIndex. (#145) 2023-06-03 11:05:35 -07:00
gsilvestrin d0c47e3838 added projection api for nodejs (#140) 2023-06-03 10:34:08 -07:00
gsilvestrin 3e14b357e7 add openai embedding function to nodejs client (#107)
- openai is an optional dependency for lancedb
- added an example to show how to use it
2023-06-01 10:25:00 -07:00
Lei Xu 306ada5cb8 Support S3 and GCS from typescript SDK (#106) 2023-05-30 21:32:17 -07:00
gsilvestrin d3aa8bfbc5 add embedding functions to the nodejs client (#95) 2023-05-26 18:09:20 -07:00
gsilvestrin f923cfe47f add create index to nodejs client (#89) 2023-05-24 16:45:58 -06:00
gsilvestrin 06cb7b6458 add query params to to nodejs client (#87) 2023-05-24 15:48:31 -06:00
gsilvestrin bdef634954 bugfix: string columns should be converted to Utf8Array (#94) 2023-05-23 14:58:49 -07:00
gsilvestrin e28fe7b468 nodejs append records api (#85) 2023-05-18 15:13:57 -07:00
gsilvestrin 395c7460d5 nodejs create_table (#75) 2023-05-15 19:00:17 -07:00
gsilvestrin 648f8123ca Exposing limit parameter (#73) 2023-05-11 09:12:06 -07:00