mirror of https://github.com/lancedb/lancedb.git synced 2025-12-22 21:09:58 +00:00

Files

msu-reevo cc81f3e1a5 fix(python): typing (#2167 )

@wjones127 is there a standard way you guys setup your virtualenv? I can
either relist all the dependencies in the pyright precommit section, or
specify a venv, or the user has to be in the virtual environment when
they run git commit. If the venv location was standardized or a python
manager like `uv` was used it would be easier to avoid duplicating the
pyright dependency list.

Per your suggestion, in `pyproject.toml` I added in all the passing
files to the `includes` section.

For ruff I upgraded the version and removed "TCH" which doesn't exist as
an option.

I added a `pyright_report.csv` which contains a list of all files sorted
by pyright errors ascending as a todo list to work on.

I fixed about 30 issues in `table.py` stemming from str's being passed
into methods that required a string within a set of string Literals by
extracting them into `types.py`

Can you verify in the rust bridge that the schema should be a property
and not a method here? If it's a method, then there's another place in
the code where `inner.schema` should be `inner.schema()`
``` python
class RecordBatchStream:
    @property
    def schema(self) -> pa.Schema: ...
```

Also unless the `_lancedb.pyi` file is wrong, then there is no
`__anext__` here for `__inner` when it's not an `AsyncGenerator` and
only `next` is defined:
``` python
    async def __anext__(self) -> pa.RecordBatch:
        return await self._inner.__anext__()
        if isinstance(self._inner, AsyncGenerator):
            batch = await self._inner.__anext__()
        else:
            batch = await self._inner.next()
        if batch is None:
            raise StopAsyncIteration
        return batch
```
in the else statement, `_inner` is a `RecordBatchStream`
```python
class RecordBatchStream:
    @property
    def schema(self) -> pa.Schema: ...
    async def next(self) -> Optional[pa.RecordBatch]: ...
```

---------

Co-authored-by: Will Jones <willjones127@gmail.com>

2025-03-10 09:01:23 -07:00

1.7 KiB

Raw Blame History

Contributing to LanceDB Python

This document outlines the process for contributing to LanceDB Python. For general contribution guidelines, see CONTRIBUTING.md.

Project layout

The Python package is a wrapper around the Rust library, lancedb. We use pyo3 to create the bindings between Rust and Python.

src/: Rust bindings source code
python/lancedb: Python package source code
python/tests: Unit tests

Development environment

To set up your development environment, you will need to install the following:

Python 3.9 or later
Cargo (Rust's package manager). Use rustup to install.
protoc (Protocol Buffers compiler)

Create a virtual environment to work in:

python -m venv venv
source venv/bin/activate
pip install maturin

Commit Hooks

It is highly recommended to install the pre-commit hooks to ensure that your code is formatted correctly and passes basic checks before committing:

make develop # this will install pre-commit itself
pre-commit install

Development

Most common development commands can be run using the Makefile.

Build the package

make develop

Format:

make format

Run tests:

make test
make doctest

Run type checking:

make typecheck

To run a single test, you can use the pytest command directly. Provide the path to the test file, and optionally the test name after ::.

# Single file: test_table.py
pytest -vv python/tests/test_table.py
# Single test: test_basic in test_table.py
pytest -vv python/tests/test_table.py::test_basic

To see all commands, run:

make help

1.7 KiB Raw Blame History