mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 14:16:54 +00:00
b5c5ede39faabb2b5ba216391b942e9bec66ce26
294
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Xuanwo
|
edc49e60e2 |
feat: expose Function failure tracebacks (#4861)
Function refresh failures and skipped-row diagnostics lose the traceback when decoded by the SDK. Expose the server-provided optional `traceback` on job failure details and Function error records in Rust, Python, and Node.js, while keeping existing compact error messages. Responses from older servers remain supported when the field is absent. Tracebacks may be truncated by the server. The Rust `Error::JobFailed` variant now stores `Box<JobFailure>` so the added diagnostics do not enlarge every `Result` error payload. |
||
|
Xuanwo
|
deb8834b5a |
fix: expose stable row IDs in Function errors (#4841)
Function error records expose physical row locations, which can change when a table is compacted. Expose the server-provided optional stable `row_id` through Rust and Python so callers can locate the same row after compaction. Document stable-ID retries and retain physical addresses for tables without stable row IDs. Older server responses continue to deserialize with `row_id=None`. The Python remote response test and Rust remote deserialization test passed on verification SDK `4e1725fa352035e2079dfd235b8271cd05adf433`, including decoding a legacy response without the new field. This branch applies the same SDK patch to current main; intervening main changes leave the affected error types and assertions unchanged. Formatting and lint checks passed after correcting the retry examples to use the `table.name` property. In a separate Linux server experiment, a real skipped Function row retained `_rowid=506` after compaction changed its physical address. The SDK field requires a server that emits stable row IDs; this PR does not change server dependencies. |
||
 Yang Cen
|
2dedde9b6c |
feat(python)!: expose typed query errors and request metadata (#4832)
Remote SQL failures currently become generic runtime errors, and HTTP table lookup can discard server error metadata. Preserve transport status, current request ID, SQL job ID and worker request ID across Rust and Python. Select typed column, table, reference and schema exceptions only when an explicit server code matches the transport status. Remote `HttpError` and `RuntimeError` handlers remain compatible. Failed SQL jobs retain `JobFailedError` with a typed SQL cause when available. Missing metadata stays absent; message text never determines the exception class. Local missing-column errors retain their type and qualified-field suggestions, and `JobFailure.source` documents structured remote Flight SQL failures. This branch integrates main at `a9ad4997`, including its released Lance 15.0.0-beta.9 dependency update and remote optimize support. The only textual conflict was two independent additions to `test_remote_db.py`; both test groups and all original assertions are retained. The feature has no dependency or lockfile delta against current main. SDK runtime validation on `03ee633a` (runtime source unchanged in `f3f2d0b8`), macOS ARM64, pinned Rust 1.97.0 and the frozen uv Python 3.14 environment: - Rebuilt the native extension from the integrated source. All 22 targeted Python tests passed: native local errors, HTTP/Flight metadata, legacy sync/async table-not-found behavior, and the three main remote-optimize regressions. - All 52 Rust SQL, metadata/source and nested-field tests passed, including the new main optimize cases. - Root and Python-binding Rust Clippy with `-D warnings`, Rust formatting, Python package Ruff and conflict-file formatting passed. - Repository-wide Ruff reports 59 existing documentation findings, and its format check identifies six documentation files. Every affected file is byte-identical to current main; these checks are not reported as passing. - Direct integration review and AST comparison confirm both conflicting test groups were preserved. Earlier exception-constructor/pickling evidence remains applicable to the unchanged Python error modules. The unlocked build exhausted disk again on a runner with the same label but only 9.4 GiB free on a 72 GiB root filesystem. Keep the existing runner and dependency caches, serialize Cargo compilation, and remove only completed test/bench executables reported by Cargo. This job never runs those binaries; normal binaries, libraries and build-script outputs remain available. Fresh resolution, the ci profile, all features, and test/bench compilation and linking are retained. Three probes using Cargo and target-path validation pass with `-D warnings`: all fixture targets link while ordinary outputs remain usable; an undefined symbol still produces Cargo exit 101 and its linker diagnostic; files outside the target directory are never removed. The [current-head build-no-lock job](https://github.com/lancedb/lancedb/actions/runs/41911613851/job/126860364065) passed on `f3f2d0b8`: all compilation/linking completed, 20 completed test executables were removed, and `target` retained about 7.3 GiB. This successful run had a 145 GiB root with about 78 GiB free afterward; it was not an identical rerun of the failed 72 GiB image. The driver tests also passed in this job. All previous-head jobs ended naturally before the push; new-head CI and review gates still apply. Typed remote input exceptions require the server metadata contract. Older servers retain generic transport errors and existing missing-table compatibility. The optimize tests use mock servers and do not establish downstream server acceptance. BREAKING CHANGE: Rust callers with exhaustive matches on `lancedb::Error` must handle the new `FieldNotFound` and `Sql` variants. Local `Error::ColumnNotFound` failures now raise `lancedb.exceptions.ColumnNotFoundError`, a `ValueError` subclass. Python callers that previously caught only `RuntimeError` for this local failure must catch `ColumnNotFoundError` or `ValueError`. |
||
  Bruno RamirezandClaude Sonnet 5
|
a9ad499781 |
feat: support remote table compaction via SQL OPTIMIZE TABLE statement (#4802)
`Table::optimize()` previously returned `NotSupported` for remote tables. This PR wires it to the LanceDB Enterprise server's `OPTIMIZE TABLE ... REWRITE DATA` SQL statement, submitted over Flight SQL. Only `OptimizeAction::Compact` without a custom `remap_options` or non-default `CompactionOptions` is supported remotely. `OptimizeAction::All`, `::Prune`, and `::Index` all return a clear `NotSupported` error instead of silently doing the wrong thing. A branch-scoped table handle submits a `WITH (branch = '<branch>')` clause; the statement doesn't validate the branch exists itself, so an unknown branch surfaces as an async job failure rather than an error from this call. The server's SQL job adapter doesn't expose compaction metrics, so `OptimizeStats::compaction` is `Some(CompactionMetrics::default())`, a placeholder rather than real numbers — it must still be `Some`, since callers (including the Python/Node bindings) unwrap it unconditionally on success. This mirrors how `refresh_materialized_view_async` already handles the same limitation for its own metrics. The Python and Node `optimize()` convenience methods call a new shared `Table::optimize_full`, which runs `Compact` and then, on a remote table, skips `Prune` and `Index` instead of attempting and failing on them — returning zeroed prune stats rather than submitting a compaction job and then raising. Python's `RemoteTable.optimize()` also now forwards `compaction_options` to the Rust layer, so a non-default value is rejected before submission instead of being silently dropped. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Xuanwo
|
b09735529b |
feat(sql): return durable materialized view job handles (#4824)
Materialized-view creation and refresh can outlive the submitting connection, but their SQL query IDs cannot be reopened through the Job Registry. On servers that negotiate early handles, return the durable registry job ID and use the existing job API to describe, reopen, wait for, or cancel it. Older servers retain the connection-scoped query handle. Registry refresh handles forward running and terminal progress observations and establish the committed-version read-freshness fence before wait returns. Preserve a nested SQL failure ID separately from the outer job ID through Rust and the Python native bindings, so Function failures remain traceable. Rust callers constructing `JobFailure` directly must supply the new optional `sql_job_id` field or use `..Default::default()`. Validation: seven SDK regression cases and the docs-link test pass, including plain wait and running-to-terminal progress observations; the observer regression fails before the fix. Formatting and strict Clippy pass. Earlier accepted coverage includes a real Flight test service exercising materialized-view creation followed by registry describe/reopen/cancel, and two Python tests after rebuilding the native extension. The published product tree matches the validated candidate. Refs ENT-3562. The matching Sophon server change is being validated separately before publication. |
||
|
Xuanwo
|
1544ad65a8 |
feat: expose structured job progress to SDK callers (#4808)
Job callers currently lose the server's structured progress when describing, listing, or waiting for a job. Expose Rust and Python progress snapshots and optional wait observers using the existing backend poller. Python supports callbacks, a caller-owned tqdm-like bar, or `progress=True`; `status()` still returns a string and the default wait behavior is unchanged. Preserve unknown progress fields and phases from newer servers, and report omitted counters as unknown. Each wait calibrates a caller-owned bar from its first complete snapshot, then uses a high-water mark within the same execution epoch; a new epoch or smaller plan resets the bar. ETA is best-effort and omitted for missing, stale, paused, or incompatible samples. Newly submitted index and refresh handles forward observers through their freshness wrapper while retaining successful-wait read consistency. The Jobs guide includes draft Python/Rust snapshot and observer examples for release; TypeScript progress support remains tracked under ENT-3500. Validation on the development EC2 host: the initial 11 Python progress tests, REST fixture caller, and two Rust progress regressions passed. After fixing freshness-wrapper forwarding, rebuilt the Python extension and ran two submitted-handle Python tests plus the new Rust refresh and index regressions (one test each); all passed. Strict Clippy, rustfmt, and targeted Ruff passed. The final commit only wraps the new guide section in an MDX draft and clarifies language support. Repository-wide Ruff reports 48 existing findings in six unchanged docs/web-tests files. This publishes the SDK portion of ENT-3500. Server-side live-split and terminal-progress changes and the combined server/SDK end-to-end scenario remain in progress. Refs ENT-3500. BREAKING CHANGE: Rust struct literals for `JobInfo` must add `progress: None`, `phase: None`, and `paused: false`. Literals for `JobDescription` must add `progress: None`, `phase: None`, `queue_reason: None`, `waiting_for_job_id: None`, and `paused_at_millis: None` when that metadata is unavailable. Existing Python calls remain compatible. |
||
|
Xuanwo
|
2d70bec53a |
feat: add UTM-tagged docs links to SDK warnings and enterprise-only errors (#4797)
Add a central documentation link registry (`lancedb::docs::DocLink`) in the Rust core that builds `docs.lancedb.com` URLs with UTM parameters (`utm_source=lancedb_sdk`, `utm_medium=warning|error`, `utm_campaign=<id>`, `utm_content=<sdk>-<version>`) placed before any `#anchor` fragment and without user-identifying data. - Initialize `utm_content` per language binding (`python`, `nodejs`, `cli`, defaulting to `rust`) and expose the link builder to Python so `DBConnection` and `Job` stubs reuse the core definitions. - Append published documentation links to Enterprise-only `NotSupported` / `NotImplementedError` errors across database, job, catalog, and table operations. - Emit rate-limited maintenance warnings (once per table per process when `warn` logging is enabled) when opening a native table with at least 100 small fragments (`< 100k` physical rows each) or at least 1,000,000 unindexed rows on an optimizable user index, reading only manifest metadata without opening data or `_indices/` files, and document the thresholds in the compaction and reindexing guides. - Verify published/in-repo `docs/web/` targets and `#anchor` fragments in unit tests and include extracted SDK links in the scheduled `docs-link-check` workflow. Linear: OSS-2772 |
||
  LeoandLeo
|
f2bcb156ad |
feat: expose approx_mode on vector queries
`ApproxMode` landed in the Rust core in #3549, but the Python and Node.js bindings were never plumbed, so `approx_mode` is unreachable from either SDK. Both were built mirroring the existing `ef` plumbing. Co-authored-by: Leo <leo@insight.co> |
||
 Srijan Keshri
|
90b2b264d9 |
feat(python): accept DataFrames and readers in add_columns
Python's `add_columns` can now accept precomputed values from pandas or Polars DataFrames, Arrow Tables, RecordBatches, readers, and iterables of batches on local tables. The sync and async APIs pass an Arrow stream to the existing Rust reader transform without materializing the whole reader. Rows attach positionally to the current live rows in storage scan order. Input must contain only new columns and exactly one row per live row; pandas indexes are ignored. Readers are consumed once, and their batch boundaries can differ from table fragments. Keep the table unchanged between computing values and adding them. SQL mappings, Function applications, computed columns, and all-null schema inputs keep their existing behavior. Data inputs cannot be combined with computed columns, and remote tables reject reader inputs with `NotImplementedError`. Tests cover deleted rows, row counts, empty batches and tables, reader errors, collisions, reserved metadata, Arrow types, and the existing input forms. Both guides describe the supported inputs and limits. Fixes #1959. |
||
 lancedb-gatefixer[bot]
|
7466dc9d1d |
fix(python): reject embedding source updates without vectors
Fixes #4761 `Table.update` does not compute embeddings, so changing an embedding source column could leave the previous embedding in place. Source-column updates now raise `ValueError` unless every associated vector column is supplied in the same call. The guard reads Python embedding metadata and native `lancedb::column_definitions` without loading embedding models. It uses Lance's field-path parser for quoted targets and covers synchronous and asynchronous local and remote tables. Supplying every associated vector column, vector-only updates, and unrelated updates remain supported. This is a breaking change for source-only updates. The API documentation and Update rows guide explain the requirement and the `merge_insert` alternative for regenerating embeddings. Validation at the reviewed head covered 44 targeted public update cases and the full Python suite (3107 passed, 162 skipped, 70 deselected). All applicable exact-head checks and workflows completed successfully. |
||
|
lancedb-gatefixer[bot]
|
ea9194a7f3 |
fix(python): retry transient job polling errors
Fixes #4773. Python job polling now retries transient connection, send, and read failures with capped exponential backoff under the caller's original timeout. If the deadline expires while disconnected, `wait()` raises `ConnectionError` with the job ID, reconnect guidance, and the original cause; after recovery, ordinary timeouts remain `TimeoutError`. Terminal job errors, cancellation, and malformed responses continue to surface immediately. Validation: exact-head CI passed across Rust all-features, Python on Linux, macOS, and Windows, NodeJS, deny, and packaging checks. The hosted macOS wheel suite passed 3049 tests (162 skipped, 70 deselected), including sync and async recovery, timeout reuse, cancellation, terminal errors, and error preservation. |
||
  TaylorandBrendan Clement
|
9e65bd2c27 |
docs: use Enterprise terminology across website and SDK documentation (#4787)
Use “LanceDB Enterprise” throughout the staged website guides, SDK reference sources, generated API pages, and user-facing SDK messages. A separate branded “LanceDB Distributed” name adds terminology without clarifying behavior; describe where operations run directly. - Update `docs/web`, including the architecture diagram title and alt text. Use Enterprise in headings, table labels, and prose; simplify redundant product/execution-model wording and name the product only once in the diagram alt text. - Update Python docstrings and the Python/Java reference pages, TypeScript comments and native declaration comments, and Rust API documentation. Regenerate the TypeScript reference, including 24 replacements across six pages. - Update SDK warnings/errors and existing test expectations to match, along with contributor guidance and Vale rules. Remove the obsolete rule distinguishing Enterprise from Distributed. - Keep lowercase “distributed” for technical descriptions. API identifiers, anchors, asset paths, and execution behavior are unchanged. No branded-name exceptions are proposed. Validation: - Python and Node native builds succeeded; focused Python tests passed (36 passed, 4 skipped), and focused Node tests passed (14). - TypeDoc generation and the complete MkDocs reference build passed. MkDocs retains an existing unresolved `lancedb.functions.SecretReference` cross-reference. - Rust formatting, Node lint/format, changed-file Python lint/format, 32 Vale tests, and whitespace checks passed. Repository-wide Python lint has 48 existing findings in six untouched files. - Combined documentation assembly passed. Mintlify validation and link/anchor/redirect checks previously passed for both site builds; no code snippets changed. Companion PRs: [saved editions, terminology guide, and shared rules](https://github.com/lancedb/docs/pull/413), [Enterprise overview and writing rules](https://github.com/lancedb/sophon/pull/9511). --------- Co-authored-by: Brendan Clement <brendan.clement@lancedb.com> |
||
 SHAIK SHIVAJI
|
45edaa350e |
feat(python): expose code analyzer options on FTS and tokenize
Part of #4364 (Python side; the TypeScript `FtsOptions` change will follow in a separate PR). Lance already supports the code analyzer, but `FTS` and `lancedb.tokenize()` had no way to set it, so `FTS({'analyzer': 'code'})` and `FTS(split_identifiers=True)` raised `TypeError`. ## Changes - `FTS` and `lancedb.tokenize()` gain `analyzer` (`"text"` or `"code"`), `split_identifiers`, `split_on_numerics`, `preserve_original` and `index_operators`. They default to `None`, which keeps the tokenizer's own defaults, so existing indexes and calls are unchanged. - Validation follows pylance: `analyzer="code"` with a `base_tokenizer` other than `"code"` or the default `"simple"` is rejected, `analyzer="text"` with `base_tokenizer="code"` is rejected, and turning on a code flag without the code analyzer is rejected. Passing a flag as `False` is always allowed. - `analyzer="code"` with the default `base_tokenizer="simple"` selects the code tokenizer. `FTS` can't tell an explicit `"simple"` from the default, so that case is accepted too. - The deprecated `create_fts_index` is left alone; it points users at `create_index(config=FTS(...))`. - Rust `FtsIndexBuilder` is Lance's `InvertedIndexParams`, so the Rust SDK already had these options. The remote client serializes the same struct, so the options reach LanceDB Distributed in the `create_index` request body. A new case in the Rust `test_create_index` checks that body. ## Tests - `python/tests/test_fts.py`: tokenizer output for each flag, validation errors for `tokenize` and `FTS`, `False` flags on a text index, and local sync and async indexes that match `http`, `request` and `parseHTTPRequest` with splitting on and not match subwords with it off. - `cargo test --features remote remote::table::tests::test_create_index` |
||
|
Xuanwo
|
63aa06ef7d |
feat(python): expose Function GPU classes and placement (#4791)
## Summary - Add sync/async `DBConnection.list_gpu_classes()` returning typed `GpuClass` records for `GET /v1/gpu/classes`. Records include the class name, Kubernetes resource, GPU model and memory, per-worker GPU maximum, and worker CPU/memory/ephemeral-storage limits. - Add `udf(gpu_class=...)` for immutable Function-level GPU placement. It requires `gpu=True`; Function versions expose their registered `gpu_class`. - Add `gpu_class` and `gpu_count` to `refresh_column` / `refresh_column_async` and Rust `RefreshColumnOptions`. Both are sent as top-level `backfill_column` fields and validated before any request. `gpu_count` is the aggregate GPU budget across Function workers. - Validate GPU class names with the server lowercase 63-character DNS-label rule. - Document the discovery-to-registration flow, configured hardware versus free capacity, System Usage permission, and per-refresh GPU placement in the API and web guides. Older servers without `/v1/gpu/classes` produce a clear `NotImplementedError` instead of a bare 404. Part of ENT-3470. The client follows the in-flight Sophon contract on `xuanwo/ent-3470-fix` for GPU class discovery and backfill placement options. ## Breaking change `PythonRuntimeSpec::PythonV2` adds `gpu_class`, and `RefreshColumnOptions` adds `gpu_class` and `gpu_count`. Rust callers constructing either value with all fields explicitly must supply the new fields. ## Validation At `7685c968`: Rust GPU tests (5 passed), Rust/PyO3 clippy, Python extension build, pyright, ruff, and Python Function/runtime, discovery, refresh placement, and refresh regressions (147 passed, 6 skipped). At `b22428fd`: Rust GPU tests (5 passed), rustfmt, ruff, pyright, Function slice 2 (85 passed), and remote GPU/refresh tests (20 passed). The final commit only updates documentation and Rust doc comments. |
||
|
Xuanwo
|
6a255926d7 |
fix(python): improve function failure diagnostics (#4790)
## Summary - Make `AsyncJob.wait(timeout)` and `Job.wait(timeout)` raise a descriptive `TimeoutError` containing the job ID and elapsed timeout. Registry jobs explain that work continues on the server and include the `db.open_job(id).wait()` reattach call. SQL query handles instead say to keep the original handle because query IDs cannot be reopened through the registry jobs API. - Render `JobFailureInfo` and `FunctionErrorRecord` with Python values instead of Rust `Option` wrappers. - Decode and expose the server's `row_address` on `FunctionErrorRecord`, and document how to retry recorded rows with `REFRESH COLUMN ... WHERE _rowaddr IN (...)` before compaction or rewrite. - Warn when an import inside a packaged Function body resolves to undeclared local source. The `udf` documentation explains why arbitrary base-image coverage and distribution-to-import-name mapping cannot be validated reliably by the SDK. `row_address` is optional, so records from older servers still decode with `None`. Rust code that constructs `FunctionErrorRecord` literals must now supply this new field. Part of ENT-3448 and ENT-3442. This consumes the server-side diagnostics from lancedb/sophon#9509 and logical row addresses from lancedb/sophon#9512. ## Validation - Rust `function_errors` remote decoding and legacy-missing-`row_address` tests. - Rust/PyO3 clippy for `lancedb` and `lancedb-python`. - Python extension build, pyright, and ruff. - Targeted Python job, Function packaging, Function error, and job detail tests: 62 passed, plus timeout-handle tests: 3 passed. |
||
|
Xuanwo
|
9786f2b63b |
fix(python): surface typed remote resource errors (#4789)
## Summary - Parse structured `error_code` and `resource` fields from remote HTTP error bodies and carry them through Rust/PyO3. - Add `SecretNotFoundError`, `FunctionNotFoundError`, `NamespaceNotFoundError`, and `AlreadyExistsError` as `HttpError` subclasses. - Expose `error_code`, `resource_kind`, `resource_name`, and `namespace_path` on HTTP exceptions while preserving `status_code`, `request_id`, and existing `except HttpError` handlers. - Do not retry structured domain errors that would otherwise become `RetryError` (notably `already_exists` on 409). Older servers that do not provide the structured fields continue to raise a plain `HttpError`. ## Compatibility This consumes the stable resource error contract added in lancedb/sophon#9516 (merge commit `35858561b3`). The existing `TableNotFoundError` behavior is unchanged. Part of ENT-3373. |
||
 BHAWESHBHASKAR
|
a9146d536a |
feat(python): expose the NGram scalar index
## What kind of change does this PR introduce?
Feature. Exposes the NGram scalar index in the Python SDK.
## What is the current behavior?
The NGram scalar index already exists in the Rust core (`Index::NGram`) and is accepted over the remote protocol (the `"NGRAM"` type), but the Python SDK has no way to create it. `lancedb.index` offers `BTree`, `Bitmap`, `LabelList`, `Fm`, and the vector indexes, but not `NGram`, so it cannot be built through `create_index(config=...)`.
## What is the new behavior?
Adds an `NGram` config to `lancedb.index` and wires it through the pyo3 binding:
```python
from lancedb.index import NGram
await table.create_index("text", config=NGram())
```
The NGram index accelerates substring filters such as `contains(col, 'needle')`, along with some `LIKE` and regular-expression predicates, on UTF-8 string columns. It is parameter free and uses Lance's default trigram parameters.
Changes:
- `lancedb.index`: new `NGram` dataclass, added to `__all__`.
- pyo3 binding (`python/src/index.rs`): a match arm mapping `NGram` to `LanceDbIndex::NGram`, plus the import and the error message.
- Type hints: `NGram` added to the `create_index` config unions and the isinstance check in `table.py`, the `IndexConfigType` union, and the `_lancedb.pyi` stubs; `"NGRAM"` added to the `IndexType` and `IndexStatistics.index_type` literals.
- Test: `test_create_ngram_index` covers creation, `list_indices`, `index_stats`, and that a substring filter uses the index.
- Docs: an NGram section in the scalar index guide.
This mirrors the approach used for the RTree index in https://github.com/lancedb/lancedb/pull/4555 (that PR keeps the RTree scope; this one adds the remaining parameter free string index).
|
||
|
lancedb-gatefixer[bot]
|
71b23ed9fb |
fix(python): preserve request metadata for missing remote tables
Fixes #4750 Remote missing-table errors retained HTTP metadata in Rust but the Python binding converted them to a plain `ValueError`, discarding the request ID and status. This raises a `TableNotFoundError` that preserves the original message, request ID, status code, and chained HTTP cause while remaining compatible with existing `ValueError` and `lance_namespace.errors.TableNotFoundError` handlers. Local and non-HTTP errors retain their previous behavior. Validation covers sync and async opens across cached/uncached and root/nested paths, exception pickling, local compatibility, the remote and error test suites, Clippy, formatting, Ruff, and all applicable PR checks including CodeQL. |
||
 Yui Numazawa
|
faa8953d81 |
fix(python): allow attribute access on namespace request dicts
Allows Python namespace implementations to read request dictionary entries as attributes while preserving mapping and `model_dump()` behavior. Missing dunder attributes continue to raise `AttributeError`. Adds a real namespace connection regression covering list/open operations and omitted optional fields. |
||
|
lancedb-gatefixer[bot]
|
9804f26bc3 |
fix: support batch size options for column refreshes (#4777)
Python column refresh methods rejected `batch_size`, and the Rust remote backend never included it in the backfill request. Callers could not reduce the rows passed to a slow Function to stay within the executor's per-call deadline. Add optional, keyword-only `batch_size` arguments throughout the synchronous and asynchronous Python APIs and bindings. Add compatible Rust `RefreshColumnOptions` APIs while preserving existing Rust method signatures. Reject non-positive sizes before submitting a job, send positive sizes in `backfill_column`, and omit the field when unset while retaining branch scoping. Batch size controls remote Function calls. Embedded expression refreshes accept and validate the option without changing execution, and blocking remote refreshes retain their existing unsupported behavior; remote callers use `refresh_column_async` and wait on the returned job. Validation: - Reproduced the original Python `TypeError` from the committed method. - Five focused Rust tests passed, covering request contents, omission, branch scoping, and rejection before any request. - `cargo check --quiet --features remote --tests --examples` passed. - `cargo clippy --quiet --features remote --tests --examples -- -D warnings` passed. - Nineteen targeted Python tests passed against the rebuilt extension, including synchronous and asynchronous remote requests, invalid sizes, and default-build behavior. Eight embedded refresh cases require the experimental-functions build and were skipped. - Rust formatting and Python formatting/lint checks passed, including CI's pinned Ruff version. Repository-wide Ruff reports 48 existing findings in untouched documentation files; none are in changed files. Fixes #4776 <!-- lance-gatekeeper-fix:v1 agent=4906ff3f5f7797a1d49a13803302021c generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
Bruno Ramirez
|
5f7ec4e0a2 |
feat(sdk): expose table overrides for remote tables
Expose table-level maintenance override read, update, and reset APIs for remote
Rust and Python tables. Local tables continue to return `NotSupported`.
Updates recursively merge the caller-provided JSON object into the current
override document so unrelated and unknown service fields are preserved. Reset
uses the explicit reset endpoint.
The integrated change keeps current main's remote database identity, namespace,
branch, SQL, WAL, and threaded mock-server behavior.
Validated with focused Rust and Python override tests and the complete applicable
CI run for
|
||
 Jack Ye
|
29691719fa |
feat: manage property graphs defined over tables
Replaces #4322, keeping only its property graph defined over tables. A property graph maps node labels and relationship types to existing tables of its namespace, in the shape of SQL/PGQ's `CREATE PROPERTY GRAPH`. It stores only that definition: a query reads the tables as nodes and edges through SQL `GRAPH_TABLE`, as they are when it runs. - **Rust:** `Connection::create_property_graph` takes a `PropertyGraphDefinition` of `NodeTable`s and `EdgeTable`s. `describe_property_graph`, `drop_property_graph` and `list_property_graphs` manage it. - **Python:** `lancedb.graph` has `NodeTable`, `EdgeTable` and `Endpoint`. Sync and async connections gain `create_property_graph`, `open_property_graph`, `drop_property_graph` and `list_property_graphs`, and a `PropertyGraph` handle describes its graph. - **Support:** remote connections only. Local connections raise `NotImplementedError` in Python and return `NotSupported` in Rust. ```python import lancedb from lancedb.graph import EdgeTable, Endpoint, NodeTable db = lancedb.connect("db://demo", api_key="...") db.create_property_graph( "social", nodes=[NodeTable("person", key="person_id", label="Person")], edges=[ EdgeTable( "knows", label="KNOWS", source=Endpoint("src_id", references=("person", "person_id")), destination=Endpoint("dst_id", references=("person", "person_id")), ) ], ) db.execute_query(""" SELECT * FROM GRAPH_TABLE (social MATCH (p:Person)-[:KNOWS]->(f:Person) WHERE p.age > 30 COLUMNS (p.name AS person, f.name AS friend)) """).read_all() ``` |
||
  ryan-haynesandRyan Haynes
|
d316cbbed4 |
fix!: support opt-in PKCE for device-code login (#4747)
## Problem and change
Some Keycloak client policies require S256 PKCE during device-code
login. Sending those parameters unconditionally breaks interoperability
with providers that expect standard RFC 8628 requests.
Configure device login with `OAuthFlow::DeviceCode(DeviceCodeOptions)`,
following the existing authorization-code options pattern.
`DeviceCodeOptions::new()` keeps PKCE disabled; `.use_pkce(true)` sends
an S256 challenge during device authorization and its matching verifier
during token polling. There is no automatic downgrade, and enabling PKCE
preserves the existing token-cache identity. Python, Node, and CLI
conversions retain ordinary device login without PKCE.
OIDC discovery and token requests now include a versioned User-Agent.
`env!("CARGO_PKG_VERSION")` is expanded at compile time and embeds the
version in the binary; no Cargo environment variable is needed when
running an installed binary or native binding.
The supported-flows guide documents the Rust-only device PKCE option and
its default.
## Rust API migration
BREAKING CHANGE: `OAuthFlow::DeviceCode` now carries
`DeviceCodeOptions`. Replace construction with
`OAuthFlow::DeviceCode(DeviceCodeOptions::new())` or
`OAuthFlow::DeviceCode(Default::default())`, and match with
`OAuthFlow::DeviceCode(_)`. Import `DeviceCodeOptions` from
`lancedb::remote` to opt into PKCE. Python and TypeScript public APIs
are unchanged.
## Validation
- `cargo +1.97.1 test --locked -p lancedb --features remote --lib
remote::`: 679 passed, including PKCE-on/off polling, option defaults,
User-Agent headers, and cache identity.
- `cargo +1.97.1 test --locked -p lancedb-python -p lancedb-nodejs --lib
test_device_code_conversion`: both native-binding conversion tests
passed.
- `cargo +1.97.1 test --locked -p lancedb --features remote --doc
remote::`: 6 passed, including the new options example and updated
session examples.
- `cargo +1.97.1 test --locked -p lancedb-cli --lib auth::tests::`: 11
passed.
- `cargo +1.97.1 fmt --all -- --check` and `git diff --check`: passed.
- A compiled version-string check ran successfully with
`CARGO_PKG_VERSION` unset at runtime.
---------
Co-authored-by: Ryan Haynes <ryan.haynes@lancedb.com>
|
||
|
Jack Ye
|
6d126a3601 |
feat: add database storage integration APIs (#4618)
Adds database-owned S3 storage integrations with Secret-style namespace
paths in a separate keyspace. Use a regular database under the cluster
root for bootstrap integrations and Secrets. `default` is an example
name, with no enforced or automatically provisioned database; `system`
remains read-only. Authentication uses either an IAM role or two
existing Secret references, including cross-database references;
configuration metadata never contains credential values.
### Python experience
```python
catalog = lancedb.connect_catalog("https://my-server.example", api_key="secret")
db = catalog.connect_database("default")
info = db.create_storage_integration(
"raw", "s3://customer-bucket/raw/",
aws_role_arn="arn:aws:iam::123456789012:role/lancedb-access",
aws_region="us-east-1",
)
print(info.configuration_instructions)
info = db.describe_storage_integration("raw")
print(list(db.list_storage_integrations()))
db.drop_storage_integration("raw")
```
For static credentials, replace `aws_role_arn` with
`aws_access_key_id_secret=SecretReference("s3_key_id",
database="default")` and
`aws_secret_access_key_secret=SecretReference("s3_secret_key",
database="default")`, imported from `lancedb.storage_integration`. Pass
`namespace_path` to address nested namespaces. Async database
connections expose the same APIs; listing is lazy. All four client
operations execute SQL through the existing SQL transport, with typed
Arrow result decoding and name-based listing cursors.
### SQL experience
```sql
CREATE STORAGE INTEGRATION default.public.raw
STORAGE_PROVIDER = 'S3'
STORAGE_PREFIX = 's3://customer-bucket/raw/'
STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::123456789012:role/lancedb-access'
STORAGE_AWS_REGION = 'us-east-1';
DESCRIBE STORAGE INTEGRATION default.public.raw;
SHOW STORAGE INTEGRATIONS IN default.public;
DROP STORAGE INTEGRATION IF EXISTS default.public.raw;
CREATE STORAGE INTEGRATION default.public.raw
STORAGE_PROVIDER = 'S3'
STORAGE_PREFIX = 's3://customer-bucket/raw/'
STORAGE_AWS_ACCESS_KEY_ID_SECRET = default.public.s3_key_id
STORAGE_AWS_SECRET_ACCESS_KEY_SECRET = default.public.s3_secret_key;
```
CREATE and DESCRIBE return configuration and a single sentence of setup
guidance. IAM mode also returns the deployment role principal, stable
external ID, customer trust policy, and prefix-scoped permissions
policy. Registration does not verify access. Creation requires namespace
CREATE_STORAGE_INTEGRATION plus parent USAGE; Secret references require
source database, namespace, and Secret USAGE. Describe requires object
USAGE; drop requires OWNERSHIP. SQL supports GRANT/REVOKE for these
objects and their creation privilege.
Includes client documentation and Flight SQL regression coverage; no
dedicated storage-integration REST endpoints are used. Local
builds/tests are excluded by request.
|
||
|
BHAWESHBHASKAR
|
891774343e |
feat(python): add CombinedFieldsQuery for BM25F full text search
Add a typed Python `CombinedFieldsQuery` for BM25F search across multiple columns. The query mirrors Lance's combined-fields semantics with ordered per-column boosts and AND/OR term matching. Convert it in both directions at the PyO3 boundary, including nested Boolean queries, and document the Python typed convenience entry. Closes #4652. |
||
|
lancedb-gatefixer[bot]
|
741cb9ad4e |
fix(python): implement remote namespace and table existence checks (#4725)
Remote Python connections inherited the unsupported-operation stubs for `namespace_exists` and `table_exists`, so both raised `NotImplementedError` even when namespace and table operations worked. Add the missing remote overrides, async methods, and PyO3 bindings using the existing namespace describe and table open paths. Only an explicit not-found result returns `False`; permission, transport, and server errors propagate. This preserves authentication, dynamic headers, retries, and nested namespace handling, and rechecks cached tables before reporting that they exist. The async namespace helper also recognizes native `NamespaceNotFound` errors so missing local namespaces return `False`. The table and namespace guides document Python remote support and distinguish it from the unsupported `exists` routes used by direct REST namespace clients. Validation: - All 40 focused regression cases pass, covering successful and missing lookups, local and remote namespaces, dynamic headers, permission/server errors, invalid table identifiers, and cached tables that disappear. - The original inherited stubs reproduce the reported `NotImplementedError`; the local namespace regression reproduced the native missing-namespace `RuntimeError` before its fix. - Full remote connection suite: 255 passed. Local namespace tests: 14 passed. - `cargo clippy --quiet -p lancedb-python --features remote --lib`, `cargo fmt --all`, Python Ruff lint, and Python formatting checks pass. - Root `ruff check .` reports 48 existing errors in documentation examples under `docs/web-tests`; the Python subtree passes and those unrelated files are unchanged. Fixes #4722 <!-- lance-gatekeeper-fix:v1 agent=4d369a0681756fdb5a8b35853d7f5607 generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
 Will JonesandClaude Opus 5.5
|
4d7a64712f |
docs: adopt Embedded/Distributed terminology across docs and SDKs (#4713)
## Summary Adopts the product terminology guide across the docs site pages, SDK docstrings and error messages. "LanceDB OSS" and local mode become **LanceDB Embedded**. "Remote" as the execution model becomes **LanceDB Distributed**, which LanceDB Enterprise provides. Where free vs paid is the point, pages contrast "LanceDB Embedded (open source)" with "LanceDB Enterprise". This is a first trial of the Embedded name, so reviewers can judge it in context. - **Embedded vs Distributed** for where execution runs (`optimize()`, consistency, branching, maintenance). Main pages define it once: "LanceDB Enterprise runs LanceDB Distributed: …" - **LanceDB Cloud** is kept only for legacy-Cloud specifics (`db://` region, the brute-force KNN limit) and marked legacy. "Cloud and Enterprise" becomes Enterprise or Distributed. - **Plan executor** becomes "executor" or the fleet name. - **Error messages** change, e.g. `optimize is not supported for remote tables.` → `... in LanceDB Distributed.` Tests that match on them are updated. - **Vale rules** (`EmbeddedName`, `OSSProduct`, `RemoteExecution`, `DistributedEnterprise`, `LegacyCloud`, `InternalNames`) enforce the terms in prose. Unchanged: identifiers (`availability.oss`, the `remote` module and feature, `RemoteTable`, `db://`), heading anchors, internal code comments, and Geneva. ## Evidence - Rust lib tests for the changed messages and the Python tests in `test_remote_db.py`, `test_materialized_views.py` and `test_table.py` pass locally. - Not run locally: - The Node tests: `remote.test.ts` and `table.test.ts` have updated assertions. - Vale. - The `experimental_functions`-marked materialized-view tests. - `docs/src` API reference was hand-edited to match the docstrings rather than regenerated. ## Merge Risk **Door:** two-way; prose and message text only. **Blast Radius:** `docs`, `error-messages`: callers matching on old error strings would break. Companion PRs: lancedb/docs (assembler labels, STYLE.md) and sophon (Enterprise pages). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
 Wyatt Alt
|
bb6aa1a147 |
feat: add a cascade option to drop_database (#4697)
The remote catalog could only drop an empty database: it always asked
the server for a restricted drop.
`DropDatabaseRequest::cascade`, `drop_database(..., cascade=True)` in
Python and `dropDatabase(name, { cascade: true })` in TypeScript ask the
server to drop everything in the database first. The name stays taken
until the server finishes cleaning up the dropped objects' storage. A
restricted drop remains the default.
|
||
|
lancedb-gatefixer[bot]andXuanwo
|
2cd635d41c |
fix: correct remote materialized view refresh results and options (#4672)
Remote materialized-view refreshes drained the SQL result and then fabricated `mode=rebuild`, `rows_written=<total view rows>`, and `source_version=0`. This made incremental refreshes report incorrect statistics. Make these three result fields optional across Rust, Python, and TypeScript, and leave them unset for remote SQL refreshes. Remove the post-refresh row count request. Local refreshes continue to populate all metrics, and `version` remains the view version observed after completion. Remote per-operation statistics remain unavailable until the SQL job adapter can retrieve them. Expose the forced-rebuild option through the Rust builder, synchronous/asynchronous Python refresh and job methods, and TypeScript refresh. Remote `full=true` sends `WITH (full = true)`; the default lets the server choose its refresh mode. Local source-version pinning and remote job cancellation are preserved. Local refresh regression tests respect the `experimental-functions` build gate, while remote refresh coverage also runs in default builds. Validation: - Rust: 20 materialized-view tests and 21 remote SQL tests passed with `remote,experimental-functions`; `cargo fmt --all` passed. - Python: rebuilt the extension in the uv-managed environment. The default build passed 23 tests with 27 feature-dependent skips. The opt-in build passed 61 tests with 3 expected default-build-only skips, covering the materialized-view suite and 14 async query-serialization cases from current main. Python Ruff lint and formatting checks passed. - TypeScript: default and opt-in native builds and TypeScript compilation passed. The default build passed 5 tests with 10 expected feature-dependent skips; the opt-in build passed all 14 materialized-view tests with the default-build-only test skipped. Lint/format checks and documentation generation passed. - Repository-wide `ruff check .` reports 48 pre-existing violations in unchanged `docs/web-tests` files; `ruff check python` passes. Fixes #4670 <!-- lance-gatekeeper-fix:v1 agent=ece43ec4b88a48d1789f60005fa3ecfc generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
lancedb-gatefixer[bot]andXuanwo
|
7fca4d666a |
fix(python): preserve async query search options during serialization (#4659)
Async `to_query_object()` dropped enabled `fast_search` on every query type and `postfilter` on plain and full-text queries. Re-executing a serialized fast search could therefore return unindexed rows. Copy enabled `fast_search` into the Python `Query` and serialize enabled plain/FTS postfilter from the Rust request's prefilter setting. Preserve the existing unset defaults: `None` for these options, with the vector query's existing `False` postfilter default. Validation: - Added 12 serialization cases across plain, FTS, and vector queries, covering both enabled and unset options, plus two FTS execution round-trip cases. Before the fix, nine cases failed, including returning an unindexed row after serialization. - Rebuilt the Python extension using the uv-managed environment. - Query tests and existing FTS fast-search tests: 192 passed, 1 slow test deselected. - Rust formatting and Python package lint/format checks pass. Repository-wide `ruff check .` reports 49 pre-existing errors in unchanged `docs/web-tests` files. Fixes #4658 <!-- lance-gatekeeper-fix:v1 agent=f828b24871a2dc928668be2de4f85078 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
lancedb-gatefixer[bot]andXuanwo
|
1edfa6077b |
fix(python): allow phrase queries without a column (#4653)
Quoted async full-text queries without `columns` carry an unset phrase column, but converting them to Python unwrapped it and raised `PanicException`. Python also required `PhraseQuery.column`, preventing callers from constructing the equivalent query object. Make the Python column optional and preserve `Option<String>` in both Rust conversion directions, leaving column selection to query planning. Regression tests cover quoted strings and structured queries with omitted, `None`, and explicit columns, query reuse, JSON serialization, slop, and document granularity. Validation: - Reproduced both missing-column failures before the fix; explicit-column controls passed. - Rebuilt the extension with `uv run --extra tests --extra dev maturin develop --extras tests,dev`. - `cd python && uv run --no-sync --extra tests pytest python/tests/test_fts.py python/tests/test_query.py::test_query_serialization_async -q`: 85 passed. - `cargo fmt --all -- --check` and `ruff check python` pass; touched Python files pass formatting checks. - `cargo clippy --quiet -p lancedb-python --lib --features remote` passes. - Root-wide `ruff check .` reports 49 existing findings in unchanged documentation examples. Fixes #4651 <!-- lance-gatekeeper-fix:v1 agent=62d36fad70c8717236283b8e97dd6b30 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
Xuanwo
|
8a4effa06a |
fix: gate experimental local functions behind an opt-in feature (#4688)
Published SDK builds currently expose local SQL computed columns and materialized-view execution that are not ready for release. Add the opt-in `experimental-functions` Cargo feature to the Rust core and forward it through the Python and Node bindings; published packages leave it disabled. Local declarations and refreshes now return `NotSupported` before mutating tables or spawning jobs, including prepared materialized-view creation. Remote Cloud/Enterprise client operations, Python UDF packaging, ordinary eager `add_columns`, and access to existing data remain available. Stored metadata and write protections stay active, and shared server commit helpers remain usable. CI covers both feature configurations. Existing local execution tests run with the feature enabled; default-build regressions cover early rejection, unchanged table versions, existing metadata protections, and normal eager transforms. API and source-build documentation explain the opt-in. Validation: Rust library tests pass with the feature disabled (1,869 tests) and enabled (1,947 tests), plus three default-build integration regressions. Python and TypeScript bindings were rebuilt and tested in both configurations, including mocked remote requests; Rust formatting, Python package lint, TypeScript lint/build, and generated docs were checked. Repository-wide Ruff still reports 48 pre-existing errors in unchanged `docs/web-tests` files. |
||
|
lancedb-gatefixer[bot]andXuanwo
|
3352bf49df |
fix(python): expose optimize compaction options (#3870)
## Summary - expose typed compaction options on synchronous and asynchronous table optimization - translate validated Python options into native Lance compaction settings - document the row-based steady-state limit and first-pass/source-budget caveats - carry the Lance beta.15 fragment-exclusion option through the typed Python API ## Root cause The native Python optimize binding always constructed default compaction settings, and the public AsyncTable API offered no way to lower the fragment row target. Wide-row fragments therefore remained compaction candidates on every later optimize call, repeatedly rewriting the accumulated table. ## Limitations This is an explicit row-based mitigation, not byte-native task sizing or a hard memory cap. Callers must derive target_rows_per_fragment from expected row size. Existing oversized fragments are rewritten whole on the first pass; max_bytes_per_file limits output file size rather than compaction memory; and source budgets admit whole planned tasks, so a budget below the first task performs no compaction. Splitting tasks by a byte budget or returning a planner error requires a Lance-side change. ## Validation - cargo check --release -p lancedb-python - cargo clippy --quiet -p lancedb-python - cargo fmt --all --check - ruff format --check python tests - ruff check python tests - uv run --no-sync pytest python/tests/test_table.py -k optimize_compaction -q (16 passed) Fixes #2325 <!-- lance-gatekeeper-fix:v1 agent=9b58eee0c79040b19540e259af0e676c generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
lancedb-gatefixer[bot]andXuanwo
|
66357275bb |
fix: inherit Lance defaults for IVF RQ quantization (#4585)
`IvfRq()` forced `num_bits=1` through Python's Rust binding, overriding Lance's five-bit default. The shared native index creation path also substituted one bit when the Rust or Node builder left the value unset. Make Python's bit count optional and forward only explicit values. Use `RQBuildParams::default()` for an unset native bit count, and omit unset values when dispatching to pylance for accelerator builds. Update the Python, Rust, Node, and vector-index documentation. New indexes with an unset bit count now inherit Lance's default, currently five bits, across Python, Rust, and Node. Five-bit vector codes use roughly five times the storage of one-bit codes, plus other index overhead. This also changes reader compatibility: indexes created with `num_bits >= 2` use a newer on-disk layout that some older LanceDB versions cannot read. Upgrade readers to versions that support multi-bit IVF_RQ, or explicitly set `num_bits=1` (for example, `IvfRq(num_bits=1)` in Python) when sharing indexes with readers limited to one-bit IVF_RQ. Validation: - Reproduced six regression failures before the fix: one-bit default metadata, rejected `None`, and incorrect remote/accelerator forwarding. - After merging `main` through `b762ec22`, rebuilt both native bindings against Lance 15.0.0-beta.1. Python validation passed with 106 tests and 15 optional Ollama skips. Coverage includes the full index suite, omitted and explicit bit counts, reading Lance-created IVF_RQ indexes, accelerator forwarding, CUDA-unavailability handling, timeout validation, remote requests, and table/namespace listing behavior. - Rust formatting and Python package Ruff lint/format checks pass. Node `pnpm lint`, `pnpm build` (including TypeScript checks), `pnpm run docs`, the full connection suite, and the existing IVF_RQ regression pass. Generated documentation adds no further changes. The merge preserves upstream's explicit RaBitQ heading anchor alongside the `IvfRq()` example and five-bit default/reader-compatibility guidance. Repository-wide `ruff check .` still reports 49 existing errors in unchanged documentation examples under `docs/web-tests`; root formatting also finds 11 unchanged documentation example files. The Python package checks pass. Fixes #4582 <!-- lance-gatekeeper-fix:v1 agent=6ef9406f0142b1ab66d53d9d56098223 generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
Jack Ye
|
dcfef7c5ab |
feat(python): open lazy blob files from SQL descriptors (#4668)
SQL blob descriptors need pinned source identity to reopen a value after
the query finishes. Adds `open_blob_file(descriptor)` and
`open_blob_files(descriptors)` so Python can read the original blob
incrementally without materializing it in the SQL result.
### Python experience
```python
rows = db.execute_query("""
SELECT f.timestamp, blob_descriptor(v.video) AS video
FROM videos v JOIN frames f ON v.id = f.video_id
""").read_all()
videos = db.open_table("videos")
for row in rows.to_pylist():
if row["video"] is not None:
with videos.open_blob_file(row["video"]) as video:
frame = extract_frame(video, row["timestamp"])
```
The singular API accepts the descriptor dictionary or an Arrow struct
scalar directly and returns a seekable `BlobFile`.
`videos.open_blob_files(rows["video"])` accepts an Arrow array/chunked
array or another iterable, opens handles as iteration advances, and
preserves order, duplicates, and nulls as `None`. Async tables provide
`await videos.open_blob_file(...)` and `async for video in
videos.open_blob_files(...)`.
The iterable API can be used directly with the Arrow result column. Zip
it with the rows to keep each frame timestamp:
```python
for row, video in zip(rows.to_pylist(), videos.open_blob_files(rows["video"])):
if video is not None:
with video:
frame = extract_frame(video, row["timestamp"])
```
Each iteration opens one handle. Null descriptors yield `None`; handles
preserve input order and duplicates, and the `with` block closes each
non-null file.
For an async table:
```python
async for video in async_videos.open_blob_files(rows["video"]):
if video is not None:
with video:
payload = await video.aread()
```
### Execution
The descriptor retains the canonical `kind`, `position`, `size`,
`blob_id`, and `blob_uri` fields and carries a `source` struct with its
dataset URI, version, commit timestamp, branch, column, row reference,
and whether that reference is a physical address. Native opening
resolves that exact snapshot without changing the table handle's
checkout and rejects a descriptor belonging to another table. Remote
handles send authenticated byte-range requests with the captured
version/branch, commit timestamp, and source URI, so an old physical
address is never interpreted against the latest table.
Known-size remote descriptors open without a payload request. External
blobs with an unspecified size use a metadata-only HEAD request. Payload
bytes load only when the file is read; full reads still allocate the
full value. Existing row-ID-based `fetch_blob_files` remains available
when a descriptor is not already present.
Coverage includes native snapshot preservation after overwrite, both row
IDs and addresses, dictionary/Arrow inputs, null/empty values, reordered
and duplicate descriptors, lazy iterable behavior, async iteration, and
remote range pinning for all six storage kinds.
Descriptor reads isolate their pinned snapshot from the live table
handle's freshness state. Both native and remote reads verify the source
commit timestamp to reject a replaced snapshot after table recreation.
|
||
|
Jack Ye
|
9af7757053 |
fix: execute remote materialized view creation through SQL (#4665)
Remote materialized-view creation currently expects a REST response containing a registry job ID, while refresh already submits SQL. Route both through the SQL query backend and preserve Python's synchronous and asynchronous return types. ### Query to Job mapping `SqlClient::submit_as_job()` submits an `ExecuteQueryRequest` through the same backend as `execute_query_async()`. Once submission succeeds, it retains the returned `Query` and starts one client-side task that consumes its result reader to completion. `Job::from_query_task()` wraps that task, its cached outcome, a single ID and a cancellation callback. Python exposes this as `Job[T]` or `AsyncJob[T]`. The existing `job_type` property maps SQL-backed handles to the new `"query"` kind. The shared `id` contains that query's UUID string, so Python callers can inspect it with `db.describe_query(job.id)` on the submitting connection. No separate `query_id`, backend enum or `type()` API is added. | Handle | `job_type` | `id` | Available lookup | | --- | --- | --- | --- | | SQL-backed | `"query"`, known at submission | Connection-scoped query UUID string | `describe_query(job.id)` | | Registry-backed | Existing server-reported kind, after `status()` or `refresh()` | Registry job ID string | Registry job APIs, descriptions and events | | Local | `None` | `None` | The current in-process handle | | Job concept | SQL-backed behavior | | --- | --- | | Submission | Await initial SQL submission before returning the handle. Submission errors propagate immediately. | | `status()` | Tracks the adapter task: running during result consumption/finalization, finished after both succeed, failed if either fails, or cancelled after cancellation. | | `wait()` | Returns the operation's typed terminal result. One task consumes the query reader once; repeated waits reuse its cached outcome without resubmitting SQL or reopening the reader. | | `cancel()` | Delegates to `Query.cancel()`. Rejected or unsupported cancellation leaves the task running; an accepted request stops the task. Cancelling a finished operation is a no-op. | | Registry details | Query-backed handles expose no registry description, event history, pause/resume or reattachment after a client restart. `refresh()` updates cached lifecycle state. | The Job tracks command completion rather than exposing SQL batches. A wait timeout does not cancel the query. Query IDs remain scoped to the submitting connection and cannot be used with registry APIs or as durable resume tokens. Python `describe_query()` accepts both UUID objects and the string `id` exposed by query-backed jobs. Rust, Python and TypeScript expose the same classification through their existing job-type accessors. ### Materialized-view operations - **CREATE:** Render `CREATE MATERIALIZED VIEW` with the view namespace and optional auto-refresh settings. After command completion, resolve the job to `None`. Synchronous creation waits on the job, then opens and returns the materialized view. - **REFRESH:** Submit `REFRESH MATERIALIZED VIEW` through the same adapter. After command completion, read the refreshed table's version and row count to construct the existing typed refresh result. Synchronous refresh waits on the same path. Remote creation requires the SQL service. Execution placement remains a server decision. REST endpoints are unchanged. The job guide documents the ID/job-type contract and query-backed limitations. Regression coverage exercises SQL submission, namespaces, refresh options, job-type/ID preservation through typed result mapping, repeated waits, failures and cancellation rejection. No local builds or tests were run. |
||
 Colin Patrick McCabe
|
631ea41a63 |
feat!: return lazy iterators for table and namespace listings (#4599)
Make table and namespace listings consistent with other resource
listings across Rust, Python, and TypeScript. Callers can iterate over
individual names across all pages, with configurable page sizes and
continuation tokens.
In order to avoid breaking existing Python users, this change adds some
compatibility methods to the returned Python iterators. Code such as:
```
for t in db.list_tables().tables:
print(t)
```
will continue to work as expected because the iterator has a `tables`
field and a `page_token` field, which function as expected to reveal the
first page of results. In other words, it is never necessary to use the
iterator __as an iterator__. It may be used as if it were an old-style
response object. (As the iterator progresses, the `tables` list may
switch to the second, third, etc. page of results.) Similar
compatibility code allows the async python API to keep working as well.
|
||
 adenzhou1350
|
3d883af674 |
fix(python): propagate invalid index wait_timeout errors (#4495)
Invalid `wait_timeout` values currently reach `Duration::extract(...).unwrap()` in both native index-creation bindings. A normal Python argument error becomes `pyo3_runtime.PanicException`, which is not caught by `except Exception`. Propagate the extraction error with `map(...).transpose()?` before constructing the index operation. Valid `None` and `timedelta` behavior is unchanged. Tests cover both `create_index` and `create_index_async`, three invalid types, valid `None`/`timedelta`, and no index created after rejection. Validation against main `0aa842b`, Python 3.12, uv-locked dependencies, freshly compiled PyO3 extension for each arm: - Unmodified source: 6 new regressions fail with `PanicException`; 6 controls pass. - Fixed source: 12/12 pass; broader index-creation regression selection: 18/18 pass. - `cargo fmt --all --check`, Ruff check/format, and diff check pass. The broader selection includes the 12 cases above. No cloud, full-suite, or performance claim. Signed-off-by: Xucheng Zhou <aden1350@outlook.com> |
||
|
adenzhou1350
|
a12aef2a8c |
fix: return a Python error when querying a closed table (#4455)
`AsyncTable.query()` unwraps the native closed-table error, so querying a closed table raises `PanicException` instead of the documented `RuntimeError`. `AsyncTable.head()` reaches the same path. Propagate `inner_ref()`'s error through `PyResult` instead. Add regressions for closed `query()` and `head()`, with `count_rows()`, repeated close, and normal open-table queries as controls. Validation against separately built native bindings: - Before: 2 regression failures (`PanicException`), 2 controls passed. - After: all 4 regressions passed. - Existing query tests plus the basic database test: 102 passed, 1 skipped on each arm (Torch is not installed). - `cargo fmt --all -- --check`, Ruff, and `git diff --check` passed. The native build used the locked Cargo dependencies with `remote,lancedb/metrics-otel,pyo3/extension-module`. This is source-tree native testing, not a complete wheel-packaging or cloud-service test. Signed-off-by: Xucheng Zhou <aden1350@outlook.com> |
||
|
adenzhou1350
|
b9ef048abb |
fix: validate Python read consistency intervals (#4454)
Negative `read_consistency_interval` values currently reach Rust's
panicking `Duration::from_secs_f64` conversion. For example:
```python
from datetime import timedelta
import lancedb
lancedb.connect("./db", read_consistency_interval=timedelta(seconds=-1))
```
Validate the interval with `Duration::try_from_secs_f64` at the three
native connection entry points and return `ValueError` before
constructing the connection or async future. `None`, zero and positive
intervals retain their behavior. The raw native binding also rejects
NaN, infinity and overflow; the public Python API still accepts
`timedelta`, not floats.
Tests cover public sync/async connections, native namespace connections,
real Python namespace clients, the three raw binding entry points and
valid-value controls.
Validation on Python 3.12 / Rust 1.97:
- Same 26 regression cases against separately built native baseline and
candidate: baseline 22 failed / 4 passed; candidate 26 passed.
- Existing basic connection and background-loop regression tests pass.
- `cargo fmt --all -- --check`, Ruff check/format and `git diff --check`
pass.
The native builds used the unchanged repository `ci` profile and
Cargo.lock with `--no-default-features --features
remote,lancedb/metrics-otel,pyo3/extension-module`. Python runtime and
focused test dependency versions came from `python/uv.lock`. Source-tree
tests used a minimal distribution-version metadata fixture derived from
the unchanged Cargo/pyproject manifests; both imported module origins
and native versions were checked in fresh processes. This is not a
complete Maturin packaging, cloud backend or full Python suite result.
No GPU was used.
AI-assisted diagnosis, implementation and validation.
Signed-off-by: Xucheng Zhou <aden1350@outlook.com>
|
||
 LeonSGP
|
1e313d3c0d |
fix(python): parse decimal exponent literals exactly (#4449)
Decimal literals such as Decimal("1E-7") and Decimal("1E+3") currently
raise ValueError while building an expression. Parse the coefficient and
exponent exactly, and normalize representable negative scales to zero so
both typed filters and predicate.to_sql() queries preserve the value.
Checked arithmetic bounds the Decimal128 coefficient to 38 digits,
preserves existing scale limits, and rejects nonfinite or overflowing
values. This does not add Decimal256 literal support.
Added 21 cases covering actual Arrow-backed table queries, SQL queries,
zero, signed exponents, precision boundaries, and errors. Both Python
3.11 and 3.13 expression suites pass 123 tests; the baseline fails 10 of
the new cases. Python binding Rust tests pass 13; official make doctest
passes 77 with 6 skips; Ruff, Rustfmt, hooks, Pyright and
workspace/tests/all-features Clippy with -D warnings pass. A fresh
environment restored from the original dependency freeze passes the full
required suite: 1528 passed, 37 skipped, 68 deselected (143.51s); while
optional Torch suites have the same baseline remote mock-server ordering
failure.
Tested against f4544628; observed official main
|
||
|
Jack Ye
|
f9680806c9 |
feat(python): expose materialized view auto-refresh creation options (#4621)
Expose optional `auto_refresh_enabled` and
`auto_refresh_interval_seconds` when creating materialized views from
Python and Rust. Omitted intervals remain unset: the server determines
when to refresh.
```python
view = db.create_materialized_view(
"recent_orders", "orders",
auto_refresh_enabled=True,
auto_refresh_interval_seconds=300,
)
```
## Architecture and workflow
- Python synchronous, asynchronous, job-returning and namespace wrappers
forward both optional values through PyO3 to the Rust creation builder.
Both default to `None`; no client interval is selected.
- Remote creation calls `POST /v1/materialized_view/{id}/create` with
the defining query and only the supplied refresh properties. A `202
Accepted` response must contain a job ID. Existing job tracking waits
for the server's ordinary SQL CREATE job before synchronous creation
opens the view.
- The server validates and authorizes the CREATE, then uses the existing
SQL worker to populate the view and persist its refresh settings. The
Lance agent subsequently uses those settings to submit REFRESH SQL jobs.
- Explicit positive finite fractional seconds are preserved. An interval
alone does not enable refresh. Embedded connections retain manual
refresh, accept explicit disabled state, and reject requests for
automatic scheduling rather than ignore them.
Includes API documentation and coverage for omission, false,
fractional/invalid intervals, namespace forwarding, server job IDs and
invalid acceptance responses.
Requires the server REST-create endpoint to be deployed before SDK
release. Existing remote `with_no_data=True` remains unsupported.
|
||
|
Jack Ye
|
1564ec10c8 |
feat: add Bootstrap split strategy for sampling with replacement (#4597)
## Context
`StreamingDataset`'s shipped split strategies (`split_random`,
`split_sequential`, `split_hash`, `split_calculated`) all share one
trait: every row in the base table is assigned to exactly one split,
exactly once. They only differ in *how* that one-to-one assignment is
chosen -- none of them can express "give me a resampled stream of N
rows, allowing repeats."
That's a distinct, common need in ML training: **the bootstrap** -- the
statistical technique (Efron, 1979) of estimating a dataset's properties
by resampling it *with replacement*. It's the same sense scikit-learn's
bagging classifiers mean by `bootstrap=True`. `SplitStrategy::Bootstrap`
adds exactly this to the data loader: sample rows uniformly with
replacement from the (optionally filtered) base table, independently per
split.
## Main use cases
- **Bagging / ensemble training** (e.g. random forests): each split
trains on an independent bootstrap resample of the training set.
- **Oversampling a rare class**: filter to the minority class, then
sample it up to whatever count balances it against the majority, instead
of exhausting its small set once per epoch.
- **Fixed-size training steps independent of dataset size**: want
exactly N batches per epoch regardless of whether the table has 50 rows
or 50 million? Set `num_samples` directly instead of it being implicitly
`len(dataset)`.
- **Weighted sampling without a weight column**: filter to "eligible"
rows (e.g. a windowed-sampling workload filtering to valid window-start
positions), then bootstrap the filtered set -- an item with more
eligible rows is naturally drawn proportionally more often, with no
explicit per-row weight needed.
## Example (an actual run, not a hypothetical one)
A 5-row base table (`row_id` 0..4), sampled via `Bootstrap {
num_samples: 8, sizes: Fixed(2), seed: Some(42) }`:
```
row_id split_id
0 0
2 0
4 0
4 0 <- row 4 repeated within split 0
4 1 <- row 4 also drawn in split 1 (independent draw)
4 1
1 1
2 1
```
8 output rows from a 5-row table -- not 5. Row 4 drawn 4 times total,
row 3 never drawn. That's with-replacement sampling actually behaving
like with-replacement sampling, not an even round-robin dressed up with
a new name.
## What changed
Adds `SplitStrategy::Bootstrap` (uniform sampling with replacement,
optionally filtered, independently seeded per split -- a given split's
content doesn't depend on how many other splits there are, the same
elastic-determinism property documented for the loader's other
strategies). No new `PermutationBuilder` Rust method was needed:
`with_split_strategy` already accepts any `SplitStrategy` value
generically. Python binding:
`PermutationBuilder.bootstrap_sample(num_samples, *, ratios=None,
counts=None, fixed=None, seed=None, split_names=None)`, mirroring
`split_random`'s existing shape.
Also adds a generic `split_strategy` parameter to `StreamingDataset`'s
constructor (a callable taking/returning a `PermutationBuilder`), since
it previously could only reach `split_random`/`split_sequential` even
though `split_hash`/`split_calculated` already shipped. This unblocks
`Bootstrap` and any future strategy without further constructor changes.
|
||
|
Jack Ye
|
a4544c5472 |
feat: configure automatic refresh when declaring computed columns (#4607)
Add `auto_refresh_enabled` and `auto_refresh_interval_seconds` to the
Rust builder and Python sync/async add-column APIs for computed columns
and registered Function outputs.
Python users can configure refresh when declaring a column:
```python
table.add_columns(
computed={"total": "price * quantity"},
auto_refresh_enabled=True,
auto_refresh_interval_seconds=300,
)
```
Omit the interval to let the server determine when to refresh:
```python
table.add_columns(computed={"discounted_total": "total * 0.9"})
```
The async API accepts the same options:
```python
await async_table.add_columns(
computed={"total": "price * quantity"},
auto_refresh_enabled=True,
auto_refresh_interval_seconds=300,
)
```
For an existing registered `FunctionApplication`, the options also apply
to Function columns. Disabling automatic refresh still permits explicit
refresh:
```python
table.add_columns(
{"features": function_application},
auto_refresh_enabled=False,
auto_refresh_interval_seconds=300,
)
table.refresh_column("features")
```
Options apply to every output declared by a call. Automatic refresh is
enabled by default; when the interval is omitted, the server determines
when to refresh. An explicit interval sets a minimum delay after a
successful refresh. The interval does not force recomputation of
unchanged inputs. Invalid intervals and options on one-time transforms
are rejected before declaration.
Native declarations persist `computed_column.auto_refresh_enabled` and
an explicitly supplied `computed_column.auto_refresh_interval_seconds`
in each output field’s metadata in the same commit as the declaration.
An omitted interval is neither stored nor sent in remote requests.
Remote declarations forward the settings to Cloud/Enterprise servers.
Local tables store these settings; automatic execution requires a
supporting server with scheduling enabled.
|
||
 Dan RammerandClaude Opus 5.5
|
92825f6f13 |
refactor!: rename MemWAL APIs to RTE (realtime engine) (#4598)
## What Renames the user-facing MemWAL surface to **RTE** (realtime engine) across Rust, Python, TypeScript, and Java, including the REST wire format. This follows the LSM → MemWAL rename in #4075. | Before | After | |---|---| | `MemWalWriteSpec` / `MemWalStats` | `RTEWriteSpec` / `RTEStats` | | `enable_memwal` / `disable_memwal` / `get_memwal_spec` | `enable_rte` / `disable_rte` / `get_rte_spec` | | `flush_memwal` / `compact_memwal` / `checkpoint_memwal` | `flush_rte` / `compact_rte` / `checkpoint_rte` | | `get_memwal_stats` / `close_memwal_writers` | `get_rte_stats` / `close_rte_writers` | | `use_memwal` (query, merge insert) | `use_rte` | | TS/Java `enableMemWal`, `useMemWal`, ... | `enableRTE`, `useRTE`, ... (napi `js_name` set explicitly) | | Java `LanceDbTableMemWal` | `LanceDbTableRTE` | | `table/memwal_stats.rs` | `table/rte_stats.rs` | Docs and comments now say "realtime engine (RTE)". Text about Lance's underlying mechanism keeps "MemWAL" ("Lance's MemWAL", "MemWAL index"). **Left alone:** Lance-owned names (`MemWalIndexDetails`, `DatasetMemWalExt`, `InitializeMemWal*`, `mem_wal` paths), internal `Lsm*` names and modules, and the `does not support with_row_id` refusal text that the hybrid fallback matches on. ## Compatibility Breaking, for an experimental API. The wire format changes, so this must roll out together with the matching sophon change (lancedb/sophon#8870): - Routes: `/enable_memwal/` → `/enable_rte/`, `/disable_memwal/` → `/disable_rte/`, `/get_memwal_spec/` → `/get_rte_spec/`, `/get_memwal_stats/` → `/get_rte_stats/`, `/flush_memwal/` → `/flush_rte/`, `/compact_memwal/` → `/compact_rte/` - Response keys: `memwal_spec` → `rte_spec`, `memwal_stats` → `rte_stats` - Request field: `use_memwal` → `use_rte` on query and merge-insert bodies The SQL keyword becomes `ALTER TABLE ... SET RTE` server-side. The guides under `docs/web` will be updated separately. ## Validation - Rust: `cargo clippy --features remote --tests --examples` clean; `cargo test --features remote -p lancedb --lib`: 1533 passed - Java: `mvn test`: 37 passed (Spotless applied with JDK 11) - Python: `ruff format` / `ruff check` clean; targeted tests (`test_rte_spec`, `test_merge_insert_lsm`, `test_remote_db`, `test_wal_hybrid`, `test_query`, `test_permutation`, `test_table`): 708 passed - TypeScript: `pnpm build`; `table`, `query` tests: 606 passed; `biome check` clean; `docs/src/js` regenerated with `pnpm run docs` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Dan RammerandClaude Opus 5
|
fe34eb3549 |
refactor!: rename LSM APIs to MemWAL and align SSTable/table shard naming (#4075)
## What Renames the MemWAL surface in two passes, across Rust, Python, TypeScript, and Java, including the wire format. ### 1. MemWAL APIs drop "LSM" The write path these APIs drive is Lance's MemWAL; "LSM" described its shape rather than naming it. Turning it on and off now reads as enabling a feature rather than installing a spec. | Before | After | |---|---| | `set_lsm_write_spec` | `enable_memwal` | | `unset_lsm_write_spec` | `disable_memwal` | | `get_lsm_write_spec` | `get_memwal_spec` | | `flush_lsm` / `compact_lsm` / `checkpoint_lsm` | `flush_memwal` / `compact_memwal` / `checkpoint_memwal` | | `get_lsm_stats` | `get_memwal_stats` | | `close_lsm_writers` | `close_memwal_writers` | | `use_lsm` (query, merge insert) | `use_memwal` | | `LsmWriteSpec` / `LsmStats` | `MemWalWriteSpec` / `MemWalStats` | | Java `LanceDbTableLsm` | `LanceDbTableMemWal` | TypeScript and Java use camelCase (`enableMemWal`, `getMemWalStats`, ...); the napi methods set `js_name` so the generated names keep the `MemWal` casing. flush/compact/checkpoint keep a `_memwal` suffix so `compact_memwal` is not confused with `compact_files` / `optimize`, which compact the base table. Internal names (`LsmMode`, `LsmScanner`, the `merge::lsm` / `query::lsm` modules) are unchanged. ### 2. Stats use SSTable and table shard Aligns the stats surface with the MemWAL naming settled upstream in lance-format/lance#7943 and #7957, where the persisted unit became an **SSTable**. **Generation → SSTable.** A `GenerationStats` describes one flushed MemTable, which is an SSTable. The generation *number* is kept — an SSTable is identified by its generation — so only the noun moved. **Bucket → table shard.** Each entry is one MemWAL shard. "Bucket" names only the hash sharding *transform*, so it was already wrong for `identity` and `year` sharding, which produce shards and no buckets at all. | Before | After | |---|---| | `GenerationStats` | `SsTableStats` | | `BucketStats` | `TableShardStats` | | `stats.buckets` | `stats.table_shards` | | `BucketStats.generations` | `TableShardStats.sstables` | | `include_generation_rows` | `include_sstable_rows` | | `newest_generation` | `newest_sstable_generation` | | `outstanding_generations` | `outstanding_sstables` | Docs and comments follow suit: "seal" → "freeze", "L0" / "flushed generations" → SSTables, "bucket" (meaning a shard) → table shard. `MemWalWriteSpec::bucket`, `num_buckets`, and the `"Sealed"` shard status value are **unchanged** — those name the sharding transform or a server value, not the shard. ## Compatibility Breaking, for an experimental API. The wire format changes with the SDKs, so this must roll out together with the matching server change: - Routes: `/set_lsm_write_spec/` → `/enable_memwal/`, `/unset_lsm_write_spec/` → `/disable_memwal/`, `/get_lsm_write_spec/` → `/get_memwal_spec/`, `/flush_lsm/` → `/flush_memwal/`, `/compact_lsm/` → `/compact_memwal/`, `/get_lsm_stats/` → `/get_memwal_stats/` - Response keys: `lsm_write_spec` → `memwal_spec`, `lsm_stats` → `memwal_stats`, `buckets` → `table_shards`, `generations` → `sstables` - Request fields: `include_generation_rows` → `include_sstable_rows`; `use_lsm` → `use_memwal` on query and merge-insert bodies ## Validation - Rust: `cargo test --features remote -p lancedb --lib --tests` — 1467 lib tests and all integration test targets pass; `cargo clippy` clean - Python: rebuilt the extension; `test_remote_db`, `test_memwal_spec`, `test_merge_insert_lsm`, `test_query`, `test_permutation`, `test_table` — 655 passed; `ruff format` / `ruff check` clean - TypeScript: `pnpm build`; `table`, `merge`, `query` tests — 593 passed; `biome check` clean; `docs/src/js` regenerated with `npm run docs` - Java: `mvn test` — 37 passed (Spotless applied with JDK 11; the pinned google-java-format does not run on JDK 21) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
lancedb-gatefixer[bot]
|
a1e4dc9f91 | fix(python): align BlobFile seek and read behavior (#4290) | ||
|
lancedb-gatefixer[bot]
|
3336bdfbec | fix(remote): bound concurrent blob reads across handles (#4311) | ||
|
Colin Patrick McCabe
|
6cc4919c3c |
feat: add an authorization API (#4363)
Add an authorization API to the LanceDB catalog. This is an API that can be used to bind access control lists (ACLs) to individual catalog objects. It can also be used to manage server-side roles, and introspect principals and groups. The new API is implemented in both the Rust, Python, and async Python styles and includes list_acls, add_acl, delete_acl, list_role_bindings, add_role_binding, delete_role_binding, get_principal, and get_group. This API implements the design principle of forwards-compatibility. So unknown privilege, object, and subject types do not cause errors, but are be supported as best as possible in the library. |
||
|
lancedb-gatefixer[bot]
|
0212cfd3d6 | fix(python): normalize blob row ID inputs and errors (#4293) |