In #2845 we ported the lancedb integration in lance-namespace to
lancedb. But that is too specific to RestNamespace. We can improve the
user entry point so that we can put local mode and future version of the
Flight SQL-based LanceDB server all behind this single
`LanceDbNamespaceClientBuilder` API.
Also I renamed `namespace` to `namesapceClient` to avoid confusion with
the namespace path.
After the refactoring on both client and server side, we should have the
ability to fully use lance REST namespace to call into LanceDB cloud and
enterprise. We can avoid having a JNI implementation (which today does
not really do anything except for vending a connection object), and just
use lance-core's RestNamespace.
We will at this moment have a LanceDbRestNamespaceBuilder to allow users
to more easily build the RestNamespace to talk to LanceDB Cloud or
Enterprise endpoint.
In the future, we could extend this further to also support the local
mode through DirectoryNamespace. That will be a separated PR.
This will make it easier to see if there are underlying problems. We
should see the actual object store HTTP request error within the error
chain after this.
This shrinks the size of a local embedded build that can disable all the
default features. When combined with
https://github.com/lancedb/lance/pull/4362 and the dependencies are
updated to point to the fix, this resolves#2567 fully.
Verified by patching the workspace to redirect to my clone of lance with
the PR applied.
```
cargo tree -p lancedb -e no-build -e no-dev --no-default-features -i aws-config | less
```
The reason that lance itself needs to change too is that many
dependencies within that project depend on lance-io/default and lancedb
depends on them which transitively ends up enabling the cloud
regardless. The PR in lance removes the dependency on lance-io/default
from all sibling crates.
---------
Co-authored-by: Will Jones <willjones127@gmail.com>
Lance Release
Jack Ye
Will Jones
Vitali Lovich
Weston Pace