diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..136b7f83 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,101 @@ +LanceDB is a database designed for retrieval, including vector, full-text, and hybrid search. +It is a wrapper around Lance. There are two backends: local (in-process like SQLite) and +remote (against LanceDB Cloud). + +The core of LanceDB is written in Rust. There are bindings in Python, Typescript, and Java. + +Project layout: + +* `rust/lancedb`: The LanceDB core Rust implementation. +* `python`: The Python bindings, using PyO3. +* `nodejs`: The Typescript bindings, using napi-rs +* `java`: The Java bindings + +Common commands: + +* Check for compiler errors: `cargo check --quiet --features remote --tests --examples` +* Run tests: `cargo test --quiet --features remote --tests` +* Run specific test: `cargo test --quiet --features remote -p --test ` +* Lint: `cargo clippy --quiet --features remote --tests --examples` +* Format: `cargo fmt --all` + +Before committing changes, run formatting. + +## Coding tips + +* When writing Rust doctests for things that require a connection or table reference, + write them as a function instead of a fully executable test. This allows type checking + to run but avoids needing a full test environment. For example: + ```rust + /// ``` + /// use lance_index::scalar::FullTextSearchQuery; + /// use lancedb::query::{QueryBase, ExecutableQuery}; + /// + /// # use lancedb::Table; + /// # async fn query(table: &Table) -> Result<(), Box> { + /// let results = table.query() + /// .full_text_search(FullTextSearchQuery::new("hello world".into())) + /// .execute() + /// .await?; + /// # Ok(()) + /// # } + /// ``` + ``` + +## Example plan: adding a new method on Table + +Adding a new method involves first adding it to the Rust core, then exposing it +in the Python and TypeScript bindings. There are both local and remote tables. +Remote tables are implemented via a HTTP API and require the `remote` cargo +feature flag to be enabled. Python has both sync and async methods. + +Rust core changes: + +1. Add method on `Table` struct in `rust/lancedb/src/table.rs` (calls `BaseTable` trait). +2. Add method to `BaseTable` trait in `rust/lancedb/src/table.rs`. +3. Implement new trait method on `NativeTable` in `rust/lancedb/src/table.rs`. + * Test with unit test in `rust/lancedb/src/table.rs`. +4. Implement new trait method on `RemoteTable` in `rust/lancedb/src/remote/table.rs`. + * Test with unit test in `rust/lancedb/src/remote/table.rs` against mocked endpoint. + +Python bindings changes: + +1. Add PyO3 method binding in `python/src/table.rs`. Run `make develop` to compile bindings. +2. Add types for PyO3 method in `python/python/lancedb/_lancedb.pyi`. +3. Add method to `AsyncTable` class in `python/python/lancedb/table.py`. +4. Add abstract method to `Table` abstract base class in `python/python/lancedb/table.py`. +5. Add concrete sync method to `LanceTable` class in `python/python/lancedb/table.py`. + * Should use `LOOP.run()` to call the corresponding `AsyncTable` method. +6. Add concrete sync method to `RemoteTable` class in `python/python/lancedb/remote/table.py`. +7. Add unit test in `python/tests/test_table.py`. + +TypeScript bindings changes: + +1. Add napi-rs method binding on `Table` in `nodejs/src/table.rs`. +2. Run `npm run build` to generate TypeScript definitions. +3. Add typescript method on abstract class `Table` in `nodejs/src/table.ts`. +4. Add concrete method on `LocalTable` class in `nodejs/src/native_table.ts`. + * Note: despite the name, this class is also used for remote tables. +5. Add test in `nodejs/__test__/table.test.ts`. +6. Run `npm run docs` to generate TypeScript documentation. + +## Review Guidelines + +Please consider the following when reviewing code contributions. + +### Rust API design +* Design public APIs so they can be evolved easily in the future without breaking + changes. Often this means using builder patterns or options structs instead of + long argument lists. +* For public APIs, prefer inputs that use `Into` or `AsRef` traits to allow + more flexible inputs. For example, use `name: Into` instead of `name: String`, + so we don't have to write `func("my_string".to_string())`. + +### Testing +* Ensure all new public APIs have documentation and examples. +* Ensure that all bugfixes and features have corresponding tests. **We do not merge + code without tests.** + +### Documentation +* New features must include updates to the rust documentation comments. Link to + relevant structs and methods to increase the value of documentation. diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index aaa46f62..00000000 --- a/CLAUDE.md +++ /dev/null @@ -1,80 +0,0 @@ -LanceDB is a database designed for retrieval, including vector, full-text, and hybrid search. -It is a wrapper around Lance. There are two backends: local (in-process like SQLite) and -remote (against LanceDB Cloud). - -The core of LanceDB is written in Rust. There are bindings in Python, Typescript, and Java. - -Project layout: - -* `rust/lancedb`: The LanceDB core Rust implementation. -* `python`: The Python bindings, using PyO3. -* `nodejs`: The Typescript bindings, using napi-rs -* `java`: The Java bindings - -Common commands: - -* Check for compiler errors: `cargo check --quiet --features remote --tests --examples` -* Run tests: `cargo test --quiet --features remote --tests` -* Run specific test: `cargo test --quiet --features remote -p --test ` -* Lint: `cargo clippy --quiet --features remote --tests --examples` -* Format: `cargo fmt --all` - -Before committing changes, run formatting. - -## Coding tips - -* When writing Rust doctests for things that require a connection or table reference, - write them as a function instead of a fully executable test. This allows type checking - to run but avoids needing a full test environment. For example: - ```rust - /// ``` - /// use lance_index::scalar::FullTextSearchQuery; - /// use lancedb::query::{QueryBase, ExecutableQuery}; - /// - /// # use lancedb::Table; - /// # async fn query(table: &Table) -> Result<(), Box> { - /// let results = table.query() - /// .full_text_search(FullTextSearchQuery::new("hello world".into())) - /// .execute() - /// .await?; - /// # Ok(()) - /// # } - /// ``` - ``` - -## Example plan: adding a new method on Table - -Adding a new method involves first adding it to the Rust core, then exposing it -in the Python and TypeScript bindings. There are both local and remote tables. -Remote tables are implemented via a HTTP API and require the `remote` cargo -feature flag to be enabled. Python has both sync and async methods. - -Rust core changes: - -1. Add method on `Table` struct in `rust/lancedb/src/table.rs` (calls `BaseTable` trait). -2. Add method to `BaseTable` trait in `rust/lancedb/src/table.rs`. -3. Implement new trait method on `NativeTable` in `rust/lancedb/src/table.rs`. - * Test with unit test in `rust/lancedb/src/table.rs`. -4. Implement new trait method on `RemoteTable` in `rust/lancedb/src/remote/table.rs`. - * Test with unit test in `rust/lancedb/src/remote/table.rs` against mocked endpoint. - -Python bindings changes: - -1. Add PyO3 method binding in `python/src/table.rs`. Run `make develop` to compile bindings. -2. Add types for PyO3 method in `python/python/lancedb/_lancedb.pyi`. -3. Add method to `AsyncTable` class in `python/python/lancedb/table.py`. -4. Add abstract method to `Table` abstract base class in `python/python/lancedb/table.py`. -5. Add concrete sync method to `LanceTable` class in `python/python/lancedb/table.py`. - * Should use `LOOP.run()` to call the corresponding `AsyncTable` method. -6. Add concrete sync method to `RemoteTable` class in `python/python/lancedb/remote/table.py`. -7. Add unit test in `python/tests/test_table.py`. - -TypeScript bindings changes: - -1. Add napi-rs method binding on `Table` in `nodejs/src/table.rs`. -2. Run `npm run build` to generate TypeScript definitions. -3. Add typescript method on abstract class `Table` in `nodejs/src/table.ts`. -4. Add concrete method on `LocalTable` class in `nodejs/src/native_table.ts`. - * Note: despite the name, this class is also used for remote tables. -5. Add test in `nodejs/__test__/table.test.ts`. -6. Run `npm run docs` to generate TypeScript documentation. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 00000000..47dc3e3d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/nodejs/AGENTS.md b/nodejs/AGENTS.md new file mode 100644 index 00000000..541a8559 --- /dev/null +++ b/nodejs/AGENTS.md @@ -0,0 +1,13 @@ +These are the typescript bindings of LanceDB. +The core Rust library is in the `../rust/lancedb` directory, the rust binding +code is in the `src/` directory and the typescript bindings are in +the `lancedb/` directory. + +Whenever you change the Rust code, you will need to recompile: `npm run build`. + +Common commands: +* Build: `npm run build` +* Lint: `npm run lint` +* Fix lints: `npm run lint-fix` +* Test: `npm test` +* Run single test file: `npm test __test__/arrow.test.ts` diff --git a/nodejs/CLAUDE.md b/nodejs/CLAUDE.md deleted file mode 100644 index 541a8559..00000000 --- a/nodejs/CLAUDE.md +++ /dev/null @@ -1,13 +0,0 @@ -These are the typescript bindings of LanceDB. -The core Rust library is in the `../rust/lancedb` directory, the rust binding -code is in the `src/` directory and the typescript bindings are in -the `lancedb/` directory. - -Whenever you change the Rust code, you will need to recompile: `npm run build`. - -Common commands: -* Build: `npm run build` -* Lint: `npm run lint` -* Fix lints: `npm run lint-fix` -* Test: `npm test` -* Run single test file: `npm test __test__/arrow.test.ts` diff --git a/nodejs/CLAUDE.md b/nodejs/CLAUDE.md new file mode 120000 index 00000000..47dc3e3d --- /dev/null +++ b/nodejs/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file diff --git a/python/AGENTS.md b/python/AGENTS.md new file mode 100644 index 00000000..993e83cf --- /dev/null +++ b/python/AGENTS.md @@ -0,0 +1,19 @@ +These are the Python bindings of LanceDB. +The core Rust library is in the `../rust/lancedb` directory, the rust binding +code is in the `src/` directory and the Python bindings are in the `lancedb/` directory. + +Common commands: + +* Build: `make develop` +* Format: `make format` +* Lint: `make check` +* Fix lints: `make fix` +* Test: `make test` +* Doc test: `make doctest` + +Before committing changes, run lints and then formatting. + +When you change the Rust code, you will need to recompile the Python bindings: `make develop`. + +When you export new types from Rust to Python, you must manually update `python/lancedb/_lancedb.pyi` +with the corresponding type hints. You can run `pyright` to check for type errors in the Python code. diff --git a/python/CLAUDE.md b/python/CLAUDE.md deleted file mode 100644 index 993e83cf..00000000 --- a/python/CLAUDE.md +++ /dev/null @@ -1,19 +0,0 @@ -These are the Python bindings of LanceDB. -The core Rust library is in the `../rust/lancedb` directory, the rust binding -code is in the `src/` directory and the Python bindings are in the `lancedb/` directory. - -Common commands: - -* Build: `make develop` -* Format: `make format` -* Lint: `make check` -* Fix lints: `make fix` -* Test: `make test` -* Doc test: `make doctest` - -Before committing changes, run lints and then formatting. - -When you change the Rust code, you will need to recompile the Python bindings: `make develop`. - -When you export new types from Rust to Python, you must manually update `python/lancedb/_lancedb.pyi` -with the corresponding type hints. You can run `pyright` to check for type errors in the Python code. diff --git a/python/CLAUDE.md b/python/CLAUDE.md new file mode 120000 index 00000000..47dc3e3d --- /dev/null +++ b/python/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file