lancedb

2,726 Commits 368 Branches 556 Tags

2 Commits

Author	SHA1	Message	Date
Justin MillerandClaude Opus 4.7	4bccb43e56	fix(python): route sync BaseQueryBuilder.to_batches through async path (#3425 ) ## Summary Fixes #3424. `LanceTakeQueryBuilder.to_batches()` raised `AttributeError: 'AsyncTakeQuery' object has no attribute 'execute'`. The inherited `BaseQueryBuilder.to_batches` called `self._inner.execute(...)`, but `self._inner` is an `AsyncQueryBase` (Python wrapper) — only its native inner exposes `execute`. Every other sync builder overrides `to_batches`, so the bug only surfaced on take-query builders, which inherit the base unchanged. `take_offsets(...).to_batches()` is broken for the same reason. Route the sync wrapper through the async `to_batches` on the background event loop, so the native `execute` is invoked from inside an awaiting context (matching how the async path works correctly). ## Repro ```python import lancedb, pyarrow as pa, tempfile db = lancedb.connect(tempfile.mkdtemp()) tbl = db.create_table("t", data=pa.table({"a": list(range(100))})) tbl.take_row_ids([0, 1, 2]).to_arrow() # works tbl.search().to_batches() # works list(tbl.take_row_ids([0, 1, 2]).to_batches()) # AttributeError (before) ``` ## Test plan - [x] New regression test `test_take_queries_to_batches` covers `take_offsets(...).to_batches()`, `take_row_ids(...).to_batches()`, and the `select(...)` projection — all fail on `main` with the patch reverted, all pass with the fix. - [x] `test_take_queries`, `test_query_builder_batches`, and `test_query_schema` still pass. - [x] `ruff format --check` and `ruff check` clean on changed files. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-21 12:11:13 -07:00
Justin MillerandClaude Opus 4.7	5b45e44ce3	fix(rust): map lance-namespace errors to TableNotFound / TableAlreadyExists (#3385 ) ## Summary `LanceNamespaceDatabase::open_table` and `create_table` were squashing `NamespaceError::TableNotFound` and `TableAlreadyExists` into generic `Error::Runtime`, so callers couldn't distinguish a missing-table or duplicate-table error from any other internal failure. Downstream this surfaced to geneva-style code as HTTP 500 / "internal server error" on operations that should have been 400/404 — see [ENT-1235](https://linear.app/lancedb/issue/ENT-1235/fix-ns-errors-for-create-tableopen-table). This PR walks the boxed-error chain from `lance::Error::Namespace` down to the inner `NamespaceError` and maps its `ErrorCode` onto the proper `lancedb::Error` variant: - `NamespaceError::TableNotFound` → `Error::TableNotFound { name, source }` - `NamespaceError::TableAlreadyExists` → `Error::TableAlreadyExists { name }` - everything else → `Error::Runtime` (unchanged behavior for the long tail) It also replaces the existing `e.to_string().contains("already exists")` string match in `LanceNamespaceDatabase::create_table` with a downcast on the `NamespaceError` code. That string-match happened to work for the `dir` backend but isn't guaranteed to match the REST namespace backend's error format; the downcast works for both. The chain-walk is needed because `DatasetBuilder::from_namespace` re-wraps the inner namespace error in a fresh `lance::Error::Namespace`, so a single top-level downcast misses it. ## How this helps geneva Geneva's workaround (linked in the parent issue) currently has to use `except Exception:` with a `# todo: this is too broad` comment, plus `str(e).lower().contains("already exists")` string matching, because the namespace-impl path raised a generic `RuntimeError`. After this PR: - `db.open_table("missing")` raises `ValueError("Table 'missing' was not found")` (via the existing Python binding mapping of `TableNotFound` → `PyValueError`) — geneva can catch `ValueError` cleanly. - `db.create_table("dup")` raises `ValueError("Table 'dup' already exists")` reliably across both `dir` and REST backends, so the existing string match becomes deterministic. In phalanx (the sophon REST server), `LanceDBError::TableNotFound` and `LanceDBError::TableAlreadyExists` already map directly to HTTP 404 and HTTP 400 respectively — see [phalanx/src/error.rs:77-94](https://github.com/lancedb/sophon/blob/main/src/rust/phalanx/src/error.rs#L77). No phalanx code change is needed for the bug fix; the previous 500 came from phalanx's string-match fallback not finding `"namespace"` AND `"not found"` in the `Runtime` error's debug-formatted message. ## Follow-up [ENT-1246](https://linear.app/lancedb/issue/ENT-1246/remove-dead-namespace-error-string-matching-in-phalanx) — after this lands and phalanx picks up the new lancedb, the string-matching fallback for table errors in `src/rust/phalanx/src/error.rs` (lines 99-168, 236-256, 502-514) and `src/rust/phalanx/src/rest/table/create_table.rs` (lines 224-241) becomes dead code and can be removed. The `// TODO: Refactor for better namespace error handling` comment at phalanx/src/error.rs:96-98 is exactly what this PR addresses on the lancedb side; ENT-1246 finishes the cleanup on the sophon side. ## Test plan - [x] `cargo test --quiet --features remote -p lancedb --lib` — all 495 lib tests pass, including 4 new tests in `database::namespace::tests`: - `test_namespace_table_not_found` — extended to assert `Error::TableNotFound` (was just `is_err()`) - `test_namespace_open_table_not_found_at_root` — covers the root-namespace path - `test_namespace_create_table_already_exists` — covers child namespace - `test_namespace_create_table_already_exists_at_root` — covers root namespace - [x] `cargo clippy --quiet --features remote --tests` — clean - [x] `cargo fmt --all` — clean - [x] Manually confirmed (via test failures before the fix) that the two `open_table` tests were returning `Error::Runtime { message: "Failed to get table info from namespace: Namespace { source: TableNotFound { ... } }" }` prior to this change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-14 15:19:23 -07:00

Author

SHA1

Message

Date

Justin MillerandClaude Opus 4.7

4bccb43e56

fix(python): route sync BaseQueryBuilder.to_batches through async path (#3425 )

## Summary

Fixes #3424.

`LanceTakeQueryBuilder.to_batches()` raised `AttributeError:
'AsyncTakeQuery' object has no attribute 'execute'`. The inherited
`BaseQueryBuilder.to_batches` called `self._inner.execute(...)`, but
`self._inner` is an `AsyncQueryBase` (Python wrapper) — only its native
inner exposes `execute`. Every other sync builder overrides
`to_batches`, so the bug only surfaced on take-query builders, which
inherit the base unchanged. `take_offsets(...).to_batches()` is broken
for the same reason.

Route the sync wrapper through the async `to_batches` on the background
event loop, so the native `execute` is invoked from inside an awaiting
context (matching how the async path works correctly).

## Repro

```python
import lancedb, pyarrow as pa, tempfile
db = lancedb.connect(tempfile.mkdtemp())
tbl = db.create_table("t", data=pa.table({"a": list(range(100))}))

tbl.take_row_ids([0, 1, 2]).to_arrow()        # works
tbl.search().to_batches()                     # works
list(tbl.take_row_ids([0, 1, 2]).to_batches())  # AttributeError (before)
```

## Test plan

- [x] New regression test `test_take_queries_to_batches` covers
`take_offsets(...).to_batches()`, `take_row_ids(...).to_batches()`, and
the `select(...)` projection — all fail on `main` with the patch
reverted, all pass with the fix.
- [x] `test_take_queries`, `test_query_builder_batches`, and
`test_query_schema` still pass.
- [x] `ruff format --check` and `ruff check` clean on changed files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-21 12:11:13 -07:00

Justin MillerandClaude Opus 4.7

5b45e44ce3

fix(rust): map lance-namespace errors to TableNotFound / TableAlreadyExists (#3385 )

## Summary

`LanceNamespaceDatabase::open_table` and `create_table` were squashing
`NamespaceError::TableNotFound` and `TableAlreadyExists` into generic
`Error::Runtime`, so callers couldn't distinguish a missing-table or
duplicate-table error from any other internal failure. Downstream this
surfaced to geneva-style code as HTTP 500 / "internal server error" on
operations that should have been 400/404 — see
[ENT-1235](https://linear.app/lancedb/issue/ENT-1235/fix-ns-errors-for-create-tableopen-table).

This PR walks the boxed-error chain from `lance::Error::Namespace` down
to the inner `NamespaceError` and maps its `ErrorCode` onto the proper
`lancedb::Error` variant:

- `NamespaceError::TableNotFound` → `Error::TableNotFound { name, source
}`
- `NamespaceError::TableAlreadyExists` → `Error::TableAlreadyExists {
name }`
- everything else → `Error::Runtime` (unchanged behavior for the long
tail)

It also replaces the existing `e.to_string().contains("already exists")`
string match in `LanceNamespaceDatabase::create_table` with a downcast
on the `NamespaceError` code. That string-match happened to work for the
`dir` backend but isn't guaranteed to match the REST namespace backend's
error format; the downcast works for both.

The chain-walk is needed because `DatasetBuilder::from_namespace`
re-wraps the inner namespace error in a fresh `lance::Error::Namespace`,
so a single top-level downcast misses it.

## How this helps geneva

Geneva's workaround (linked in the parent issue) currently has to use
`except Exception:` with a `# todo: this is too broad` comment, plus
`str(e).lower().contains("already exists")` string matching, because the
namespace-impl path raised a generic `RuntimeError`. After this PR:

- `db.open_table("missing")` raises `ValueError("Table 'missing' was not
found")` (via the existing Python binding mapping of `TableNotFound` →
`PyValueError`) — geneva can catch `ValueError` cleanly.
- `db.create_table("dup")` raises `ValueError("Table 'dup' already
exists")` reliably across both `dir` and REST backends, so the existing
string match becomes deterministic.

In phalanx (the sophon REST server), `LanceDBError::TableNotFound` and
`LanceDBError::TableAlreadyExists` already map directly to HTTP 404 and
HTTP 400 respectively — see
[phalanx/src/error.rs:77-94](https://github.com/lancedb/sophon/blob/main/src/rust/phalanx/src/error.rs#L77).
No phalanx code change is needed for the bug fix; the previous 500 came
from phalanx's string-match fallback not finding `"namespace"` AND `"not
found"` in the `Runtime` error's debug-formatted message.

## Follow-up


[ENT-1246](https://linear.app/lancedb/issue/ENT-1246/remove-dead-namespace-error-string-matching-in-phalanx)
— after this lands and phalanx picks up the new lancedb, the
string-matching fallback for table errors in
`src/rust/phalanx/src/error.rs` (lines 99-168, 236-256, 502-514) and
`src/rust/phalanx/src/rest/table/create_table.rs` (lines 224-241)
becomes dead code and can be removed. The `// TODO: Refactor for better
namespace error handling` comment at phalanx/src/error.rs:96-98 is
exactly what this PR addresses on the lancedb side; ENT-1246 finishes
the cleanup on the sophon side.

## Test plan

- [x] `cargo test --quiet --features remote -p lancedb --lib` — all 495
lib tests pass, including 4 new tests in `database::namespace::tests`:
- `test_namespace_table_not_found` — extended to assert
`Error::TableNotFound` (was just `is_err()`)
- `test_namespace_open_table_not_found_at_root` — covers the
root-namespace path
- `test_namespace_create_table_already_exists` — covers child namespace
- `test_namespace_create_table_already_exists_at_root` — covers root
namespace
- [x] `cargo clippy --quiet --features remote --tests` — clean
- [x] `cargo fmt --all` — clean
- [x] Manually confirmed (via test failures before the fix) that the two
`open_table` tests were returning `Error::Runtime { message: "Failed to
get table info from namespace: Namespace { source: TableNotFound { ... }
}" }` prior to this change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-14 15:19:23 -07:00