mirror of
https://github.com/lancedb/lancedb.git
synced 2025-12-25 22:29:58 +00:00
d974413745
I know there's a larger effort to have the python client based on the core rust implementation, but in the meantime there have been several issues (#1072 and #485) with some of the azure blob storage calls due to pyarrow not natively supporting an azure backend. To this end, I've added an optional import of the fsspec implementation of azure blob storage [`adlfs`](https://pypi.org/project/adlfs/) and passed it to `pyarrow.fs`. I've modified the existing test and manually verified it with some real credentials to make sure it behaves as expected. It should be now as simple as: ```python import lancedb db = lancedb.connect("az://blob_name/path") table = db.open_table("test") table.search(...) ``` Thank you for this cool project and we're excited to start using this for real shortly! 🎉 And thanks to @dwhitena for bringing it to my attention with his prediction guard posts. Co-authored-by: christiandilorenzo <christian.dilorenzo@infiniaml.com>
LanceDB
A Python library for LanceDB.
Installation
pip install lancedb
Usage
Basic Example
import lancedb
db = lancedb.connect('<PATH_TO_LANCEDB_DATASET>')
table = db.open_table('my_table')
results = table.search([0.1, 0.3]).limit(20).to_list()
print(results)
Development
LanceDb is based on the rust crate lancedb and is built with maturin. In order to build with maturin
you will either need a conda environment or a virtual environment (venv).
python -m venv venv
. ./venv/bin/activate
Install the necessary packages:
python -m pip install .[tests,dev]
To build the python package you can use maturin:
# This will build the rust bindings and place them in the appropriate place
# in your venv or conda environment
matruin develop
To run the unit tests:
pytest
To run the doc tests:
pytest --doctest-modules python/lancedb
To run linter and automatically fix all errors:
ruff format python
ruff --fix python
If any packages are missing, install them with:
pip install <PACKAGE_NAME>
For Windows users, there may be errors when installing packages, so these commands may be helpful:
Activate the virtual environment:
. .\venv\Scripts\activate
You may need to run the installs separately:
pip install -e .[tests]
pip install -e .[dev]
tantivy requires rust to be installed, so install it with conda, as it doesn't support windows installation:
pip install wheel
pip install cargo
conda install rust
pip install tantivy