In Rust and Node, we have been swallowing filter validation errors. If
there was an error in parsing the filter, then the filter was silently
ignored, returning unfiltered results.
Fixes#1081
Arrow-js uses brittle `instanceof` checks throughout the code base.
These fail unless the library instance that produced the object matches
exactly the same instance the vectordb is using. At a minimum, this
means that a user using arrow version 15 (or any version that doesn't
match exactly the version that vectordb is using) will get strange
errors when they try and use vectordb.
However, there are even cases where the versions can be perfectly
identical, and the instanceof check still fails. One such example is
when using `vite` (e.g. https://github.com/vitejs/vite/issues/3910)
This PR solves the problem in a rather brute force, but workable,
fashion. If we encounter a schema that does not pass the `instanceof`
check then we will attempt to sanitize that schema by traversing the
object and, if it has all the correct properties, constructing an
appropriate `Schema` instance via deep cloning.
This PR adds the same consistency semantics as was added in #828. It
*does not* add the same lazy-loading of tables, since that breaks some
existing tests.
This closes#998.
---------
Co-authored-by: Weston Pace <weston.pace@gmail.com>
This PR also reworks the table creation utilities significantly so that
they are more consistent, built on top of each other, and thoroughly
documented.
A `count_rows` method that takes a filter was recently added to
`LanceTable`. This PR adds it everywhere else except `RemoteTable` (that
will come soon).
Allow passing API key as env var:
```shell
export LANCEDB_API_KEY=sh_123...
```
with this set, apiKey argument can omitted from `connect`
```js
const db = await vectordb.connect({
uri: "db://test-proj-01-ae8343",
region: "us-east-1",
})
```
```py
db = lancedb.connect(
uri="db://test-proj-01-ae8343",
region="us-east-1",
)
```
Close#773
we pass an empty table over IPC so we don't need to manually deal with
serde. Then we just return the schema attribute from the empty table.
---------
Co-authored-by: albertlockett <albert.lockett@gmail.com>
Close#773
we pass an empty table over IPC so we don't need to manually deal with
serde. Then we just return the schema attribute from the empty table.
---------
Co-authored-by: albertlockett <albert.lockett@gmail.com>
Note this currently the filter/where is only implemented for LocalTable
so that it requires an explicit cast to "enable" (see new unit test).
The alternative is to add it to the Table interface, but since it's not
available on RemoteTable this may cause some user experience issues.
https://github.com/lancedb/lancedb/issues/623
Fixes issue trying to print response status when using remote client. If
the error is not an HTTP error (e.g. dns/network failure), there won't
be a response.
Will Jones
Weston Pace
QianZhu
Rob Meng
Lei Xu
Bert
Chang She
Andrew Miracle
Aidan