mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-12 14:52:26 +00:00
6a1645116ee441855e5950369e3c9e44f54dbcd3
3115
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Brendan Clement
|
6a1645116e |
docs: declare where ingestion and table pages apply
Creating, writing, updating, merging and deleting rows, managing tables, branches and read consistency work in the embedded library and on a deployment, and those pages now say so. The SQL forms they mention need a deployment. Where the offerings differ, the section says how: a deployment reuses an existing table without comparing schemas, a merge insert can be refused past a configured number of unindexed rows, space is reclaimed by optimize() locally and by the deployment's compaction and configured cleanup otherwise, renaming depends on the catalog, a deployment clones shallowly only, drops all tables only locally, keeps freshness state per table handle and defaults freshness to its own setting, and refuses to delete a branch main depends on. table_exists works only on a namespace connection. Guarantees and limits span both offerings, so they carry section notes rather than a label: reads, retention, index freshness and compaction differ on a deployment, and the name and job limits are a deployment's. |
||
|
Brendan Clement
|
027597d36d |
docs: declare where indexing and maintenance apply
Indexes are built in the embedded library and on a deployment, which builds them in the background, keeps most of them current, and builds only IVF_FLAT, IVF_PQ, IVF_RQ and IVF_HNSW_SQ vector indexes. Compaction runs on both, by optimize() locally and in the background on a deployment; cleanup on a deployment runs only where its configuration enables it, so it varies; background jobs exist only on a deployment. Sections that described one offering as if it were both now name it: optimize() is embedded only, the brute-force limit and automatic IVF_PQ are deployment settings, FM indexes get no background updates, a remainder below the deployment's threshold can keep wait_for_index from finishing, GPU builds on a deployment are IVF_PQ with a configured GPU count, list_indices on a deployment omits num_unindexed_rows, and the background jobs page no longer says every other job is one you start. |
||
|
Brendan Clement
|
e85f1b4434 |
docs: declare where the search pages apply
Search, full-text search, hybrid search, filtering, query plans and reranking all work in the embedded library and against a deployment, and now say so. Where the two differ, the section says how: a deployment returns a query's whole result in one response, the synchronous Python remote table refuses to_pandas, to_arrow and to_lance, probe widening depends on the deployment's version, full-text indexes build in the background and are kept current without optimize(), distributed scoring can differ slightly, model files have to be present on the deployment, plans carry deployment operators, and hybrid explain_plan is Python only. Reranking runs in the application either way. |
||
|
Brendan Clement
|
05c7ba4889 |
docs: correct the labelled pages against the pinned deployment
Authorization wrote objects as namespace:db/ns and table:db/ns/table, which the deployment rejects; it writes namespace:db:ns and table:db:ns$table, nesting namespaces with $. The page now lists the view, function and secret objects, CREATE_FUNCTION and CREATE_SECRET, USAGE and UPDATE where they apply beyond tables, and what OPERATE and USAGE on system allow. An a: subject names the principal holding a key rather than a narrower grantee, creation falls back to the parent's create privilege only for an unowned database or namespace, and a listing page holds at most 100 entries. The materialized view page said a deployment's refresh, options and stable row id needs depend on version and configuration. Newer versions rebuild the whole view on every create and refresh as a job, need no stable row ids, take no refresh options, and have no REST route to create one; earlier ones differed, and their views have to be recreated. The page and the REFRESH reference now say so, without naming which version a deployment runs. Vector search no longer says it has no SQL form: some deployment versions plan ORDER BY distance(...) LIMIT k as one. |
||
|
Brendan Clement
|
f228295c34 |
docs: add the OSS and Enterprise and connection pages
basics/offerings explains the two offerings and holds the comparison the assembled site generates from the pages' availability. basics/connect shows local, object storage and Enterprise connections with the existing tested connection snippets. Both follow the landing page in Getting Started. |
||
|
Brendan Clement
|
03e573f442 |
docs: declare where four pages apply
Vector search, materialized view refresh, authentication and authorization now say in their frontmatter whether LanceDB OSS and LanceDB Enterprise have what they describe, with one sentence on how. The assembled site renders that as each page's label, its sidebar tag and its comparison row, so the hand-written badges and opening paragraphs on the two security pages go. The availability was checked against the SDK and the Enterprise server: authorization is enforced only on a deployment configured for it, and a deployment accepts OAuth tokens only with an identity provider configured. Sections now say that only the Rust SDK sets approx_mode, that a deployment may search exactly minimum_nprobes partitions, and that the SDK refuses with_no_data, like source_version, on a db:// connection. |
||
 Xuanwo
|
d3390a9ca7 |
build(java): resolve Lance preview dependencies from R2 (#4476)
Lance beta and RC artifacts are moving to `https://maven.lance.org`. Add that release repository to the Java parent POM so LanceDB can resolve explicit preview versions from it, while keeping Maven Central available for historical releases and other dependencies. Validated resolution of `org.lance:lance-core:14.0.0-beta.3` from a fresh Maven cache using the actual `lancedb-core` POM. Maven recorded the new `lance` repository as the source of both the POM and JAR. The pinned Lance version and LanceDB's own publishing workflow are unchanged. |
||
|
Brendan Clement
|
07e626ec48 |
fix: check docs web test license headers (#4474)
Remove scaffold that accidentally caused the post-merge Rust license-header check to fail. Include `docs/web-tests/**` in the PR workflow paths so future docs test changes run the license checks before merge. Validation: `cargo fmt --all`; `git diff --check`; license-header-checker v1.5.0 using all four workflow commands on a clean checkout; Cargo metadata confirms all six Rust example targets remain unchanged. |
||
 Wyatt Alt
|
6a68b279bb |
fix: refuse direct writes to materialized views (#4471)
Refuse user writes (add, insert, update, delete, merge_insert, column changes) to a table carrying `mv.definition`, so a materialized view is written only by refresh. Index, optimize, and computed-column operations stay allowed. Sophon's REST write paths are handled separately in ENT-2878. Fixes ENT-2876. |
||
 Jack YeandBrendan Clement
|
47b7babc08 |
docs: bring the open-source documentation into this repository (#4122)
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).
This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.
Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.
**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.
**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.
Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.
**Updated-head CI (
|
||
 Colin Patrick McCabe
|
de005f1d61 |
refactor!: implement lazy iterators for several APIs (#4459)
Implement lazy iterators for: - secrets - views - jobs - functions - materialized views |
||
  Yuval LifshitzandClaude Opus 5.0
|
069b2f7d31 |
feat: add delete rows to table stats (#3783)
add to table stats the number of rows that has been marked for deletion, but not yet deleted. note that when a table fragment is removed due to deletion of all of its rows, these rows are not counted. since the main use of this feature is to determine when table compaction is needed, the fact that these rows are not reported should not impact that decision (deleted table fragments don't need compaction). Fixes: https://github.com/lancedb/lancedb/issues/3761 Co-authored-by: Claude Opus 5.0 <noreply@anthropic.com> |
||
 Lance Release
|
becbda70dd | Bump version: 0.40.0-beta.13 → 0.41.0-beta.0 | ||
|
Wyatt AltandClaude Opus 5.5
|
fad3a1125a |
feat: allow Function columns to read computed columns (#4460)
Lets a registered-Function column read another computed column (SQL or Function). Resolves ENT-2780. Nothing records which rows of a computed column have been filled, so a refresh treats a null computed input as "not yet computed". It skips the row and leaves the outputs null until a later refresh, after the input is filled. Because the Function never receives a null from a computed input, that input binds a non-nullable parameter. The trade-off is that a computed input which is null by definition (a SQL expression that yields null, or a Function row skipped on error) leaves the downstream row unfilled. ## Changes - `plan_function_application` and `ensure_binding_matches_schema` now accept computed inputs and record them as non-nullable. They still refuse inputs that are: - unrecognized computed kinds, - incomplete declarations, - internal Function assignment columns. - A new public `function_input_is_computed` reports whether an input reads a computed column. A new public `computed_function_inputs` lists the inputs a server refresh must find non-null before it computes a row. - `ensure_supported_function_metadata` now refuses a schema whose computed columns read each other in a cycle. A declaration can only read columns that already exist, so only a schema supplied whole can contain one. - Dropping an upstream output while a downstream binding still reads it is refused by the existing binding guard. This is now covered by a test. The server-side refresh change (skipping rows that are not ready, clearing them on recompute) lives in sophon and follows this PR. ## Testing - `cargo test -p lancedb --features remote --lib -- computed refresh freshness schema_evolution function`: 255 passed. - `cargo clippy -p lancedb --features remote --tests -- -D warnings`: clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
 lancedb-gatefixer[bot]
|
bbc4aebe55 |
fix(python): infer vector columns for sync remote search (#4409)
Sync remote string searches ignored the table's embedding function when `vector_column_name` was omitted. `auto` silently fell back to full-text search, while `vector` and `hybrid` failed with `No embedding function for None`. Resolve vector columns from the schema before creating the query builder, using the same inference helper as local search. Automatic string queries without embedding functions select full-text search first, so text-only tables and tables with multiple unconfigured vector columns remain searchable. Vector and hybrid searches still reject ambiguous columns, infer numeric queries by dimension, and honor explicit column selection. Structured full-text queries and empty queries retain their existing behavior. Regression coverage uses embedding metadata returned by a mocked remote server rather than a mocked embedding-function lookup. It also checks automatic and explicit full-text queries on schemas with zero, one, or multiple vector columns and no embedding functions. Validation after merging `main` at `87ecfeac`: - Confirmed the original embedding and ambiguity failures before the fix, and reproduced the automatic FTS inference failures before restoring the fallback. - Remote, query, hybrid, and embedding test files through the uv-managed environment: 240 passed, 4 skipped, 9 deselected (`not slow and not s3_test`). - Python extension built with `uv run --extra tests --extra dev maturin develop --extras tests,dev`. - `ruff check .` and `ruff format .` from the repository root, using CI's Ruff version 0.9.9. - `cargo fmt --all`. Fixes #4377 <!-- lance-gatekeeper-fix:v1 agent=c167c6c89d857342e5d7415b08142589 generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
a8baecb306 |
fix: reject non-finite query vectors without panicking (#4410)
Remote vector serialization unwrapped `serde_json::Number::from_f64`, which returns `None` for NaN and infinity, causing a Tokio worker panic. Validate vectors after conversion to Float32 in the shared `nearest_to` / `add_query_vector` builder so local and remote queries consistently return `Error::InvalidInput` (mapped to `ValueError` by the Python bindings). This also rejects finite Float64 inputs that overflow Float32. Make remote serialization fallible to protect requests constructed directly, including single-vector queries and both remote batch-query paths. Validation: - All 116 query-related Rust tests passed with the `remote` feature, including the three new regression cases. - The three regression cases fail with the original production code and pass with the fix. Coverage includes NaN, positive and negative infinity, Float16/32/64 Arrow inputs, Float64 overflow, and rejection before any HTTP request. - `cargo fmt --all -- --check`, `cargo check --profile ci --locked -p lancedb --features remote --tests --examples`, and `cargo clippy --profile ci --locked -p lancedb --features remote --tests --examples -- -D warnings` passed. Fixes #4375 <!-- lance-gatekeeper-fix:v1 agent=97d6925d4af7cd09767da8b552c9bcf3 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
87ecfeaccd |
fix(python): preserve embedding functions in sync remote table creation (#4408)
Sync remote `create_table(..., embedding_functions=...)` discarded the supplied configuration and only logged an unsupported-feature warning. Schema-only creation therefore lost the metadata needed by `add()` to embed text, while text-only initial data produced no vector column. Forward the configuration to the existing async creation path, remove the stale warning, and document the parameter. The shared path stores the embedding metadata and generates vectors for initial data and subsequent additions. Add a regression test against the mocked remote HTTP API for both schema-only and text-only creation. It checks the outgoing Arrow schema metadata, the metadata returned by the remote table, inferred vector types, and generated vectors during creation and `add()`. Validation: - Both regression cases failed before the fix because the outgoing embedding metadata was missing, then passed after the fix. - `uv run --extra tests --extra dev pytest python/tests/test_remote_db.py python/tests/test_embeddings.py -q -m 'not slow and not s3_test'`: 82 passed, 4 skipped, 8 deselected. - Repository-wide `ruff check .` and `ruff format .` passed using the uv-managed environment; `git diff --check` passed. Fixes #4376 <!-- lance-gatekeeper-fix:v1 agent=a6816b1f3335438bd8ad192f36395213 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
3385dc55dc |
fix(remote): validate schema for create_table with exist_ok (#4407)
Remote `create_table` with `exist_ok` opened an existing table without validating the supplied schema, so Python and TypeScript silently accepted mismatches. Capture the request schema before consuming its data stream, compare it with the opened table's schema, and return the same `Error::Schema` and message as the local backend when they differ. The existing open-table callback continues to run before validation. Regression coverage exercises matching schemas and differences in field name, type, and nullability, for both empty and data-backed creation with cached and uncached table opens. All 12 mismatch cases failed against the original implementation; all 16 cases now pass. Validation: - `cargo test --profile ci --features remote -p lancedb --lib remote::db::tests`: 113 passed. - `cargo clippy --quiet --profile ci --features remote -p lancedb --tests --examples`: passed. - `cargo fmt --all` and `git diff --check`: passed. Fixes #4378 <!-- lance-gatekeeper-fix:v1 agent=5781a39ba124cf9094d36715730342b4 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
02ed5cf129 |
fix: require checkout before restoring remote tables (#4406)
Remote handles at the latest version have no checkout version, so
`restore()` sent `{"version":null}` and surfaced a server
deserialization error. Validate the checkout version before constructing
the request and return `Error::InvalidInput` with the same message as
local tables. This shared Rust fix covers Python sync/async and
TypeScript callers.
Add a regression test proving that restore sends no request on a fresh
handle, after `checkout_latest()`, or after a successful restore clears
checkout state. It also verifies that a valid restore sends the
checked-out version. Update the schema-cache and branch restore tests to
check out a version before restoring and require success.
Validation:
- The new regression failed against the original implementation when the
mock received a null version.
- `cargo test --locked --profile ci --features remote -p lancedb --lib
restore` — 6 tests passed.
- `cargo fmt --all` and `git diff --check` passed.
- `cargo clippy --quiet --locked --profile ci --features remote -p
lancedb --tests --examples` passed.
Fixes #4381
<!-- lance-gatekeeper-fix:v1 agent=cc67544dfcbb46d631762752d55b39e3
generation=1 -->
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
|
||
|
lancedb-gatefixer[bot]
|
6b6d55e6c7 |
fix: accept updated_rows in remote update responses (#4444)
Remote update responses report their count as `updated_rows`, but the client only recognized `rows_updated` and silently defaulted the count to zero. Python sync/async and TypeScript both use this Rust result. Accept `updated_rows` as a Serde alias while retaining `rows_updated` and compatibility with older responses. Extend the remote mock test to cover both keys, zero and missing counts, returned versions, and empty responses, with and without a predicate. Validation: - Both nonzero `updated_rows` regression cases failed before the fix (0 instead of 5); all 11 focused remote tests pass after it. - All 3 native update tests pass. - `cargo fmt --all` - `cargo clippy --quiet --profile ci --features remote -p lancedb --tests --examples` Fixes #4443 <!-- lance-gatekeeper-fix:v1 agent=423249073212c9840d96ccf6d4452905 generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
 XY Zhan
|
2bf70758a7 |
fix: read back the maintained-index selection a table was given (#4447)
`maintained_indexes` on a MemWAL table has three values: a named list, `[]` for none, and `null` for every index the table has, including ones created later. This PR makes every client read all three back as they were set. - **Rust remote client** (also used by the Python and Node SDKs) reads `null` as `None`. - **Java client** reads `null` as `null`, not as an empty list. - **Docs** for the setter, the field and the getter describe the three values the same way in Rust, Python, Node and Java. The generated JS docs are regenerated to match. ## Testing - `rust/lancedb`: 73 LSM tests pass, including one case per value. With the decoder typed as a list, the `null` case fails. - Java: two cases for `null` and `[]` on the read path, run by the Java CI job. - `cargo fmt`, `cargo clippy -D warnings` and `ruff format` are clean. |
||
|
Wyatt Alt
|
dbd86ed9b7 |
feat: let CountPushdown see through MetadataEraserExec (#4458)
BaseTableAdapter wraps every scan in MetadataEraserExec, which reported an unknown cardinality effect. Lance's CountPushdown only looks through nodes that keep every row, so a SQL COUNT(*) over the adapter always scanned, even with the rule registered. The eraser now reports Equal. with_new_children kept the old schema and panicked when the new child's schema differed; it now rebuilds from the child. |
||
 LanceDB RobotandWyatt Alt
|
17e9a657e6 |
chore: update lance dependency to v14.0.0-beta.1 (#4445)
Updates the Rust workspace Lance dependencies, Cargo lockfile, and Java lance-core to [v14.0.0-beta.1](https://github.com/lance-format/lance/releases/tag/v14.0.0-beta.1). No compatibility fixes were required. Validation passed: `cargo clippy --quiet --workspace --tests --all-features -- -D warnings`, `cargo fmt --all --quiet`, and `git diff --check`. --------- Co-authored-by: Wyatt Alt <wyatt.alt@gmail.com> |
||
|
Lance Release
|
81d52a98f8 | Bump version: 0.40.0-beta.12 → 0.40.0-beta.13 | ||
|
Jack Ye
|
7e8f0d4982 |
refactor: always fully refresh materialized views (#4373)
Simplify experimental materialized views to one output-only storage
layout and one full-rebuild refresh implementation.
## Current layout
Local materialized views currently maintain incremental-refresh state
alongside the materialized values:
- an internal `__source_row_id` column for source provenance;
- `mv.definition` plus `mv.incarnation`;
- source version/timestamp, view version, and last-refresh watermarks;
- optional `mv.staging` state for table-function sources; and
- stable-row-ID requirements on the source table.
Refresh classifies source changes, appends or evicts rows when possible,
and falls back to a rebuild for unsupported deltas. Remote
create/refresh also use separate REST mutation paths.
## New layout
An MV is an ordinary Lance backing table containing only:
- the defining query's output columns;
- its current data fragments; and
- `mv.definition`, encoded as a versioned SQL envelope such as
`{"kind":"query","format":1,"query":"SELECT ..."}`.
There is no provenance column, incarnation, source/view watermark,
staging metadata, or stable-row-ID requirement. Local refresh may still
pin a source version for reproducibility, but it always recomputes the
complete result.
Full refresh streams the new result into uncommitted fragments and then
publishes a single transaction that removes every old fragment and
installs the new set. A shared conflict token prevents concurrent
replacements from both committing. This fragment-replacement primitive
is also used by Sophon's SQL worker.
## New remote create workflow
```text
Connection::create_materialized_view(...)
-> builder renders the defining SELECT
-> remote Database receives a CreateMaterializedViewRequest
-> submit CREATE MATERIALIZED VIEW ... AS ... through Flight SQL
-> retain the SQL Query handle while the Job is running
-> consume the durable SQL-job result
-> open the created table and read mv.definition
-> return MaterializedView
```
Remote refresh similarly submits `REFRESH MATERIALIZED VIEW` through
Flight SQL. Cancelling either wrapper forwards cancellation to the
retained SQL query before aborting the local task, so the server
operation is not silently left running.
## API changes
- remove incremental/no-op modes and the `full` switch;
- remove distributed grouped/partitioned refresh APIs and native
vector-source MV APIs;
- keep full-refresh results across Rust, Python, and TypeScript;
- preserve list/describe/drop as object-management operations while
routing create/refresh mutations through SQL.
The storage contract is experimental and has no compatibility shim;
views using the removed layout must be recreated.
|
||
|
Colin Patrick McCabe
|
d864e7198f |
feat!: list_databases returns an iterator (#4442)
CatalogConnection.list_databases should return an iterator so that users don't have to manually re-implement pagination. For those who do want more control over pagination, add num_page_results and page_token to the Iterator object, and also allow the iterator object to be created at a specific page token. |
||
 ccmao1130
|
f4544628ba |
fix: add UTM to reverie link, remove cloud in banner (#4446)
- Add UTM to Reverie link for tracking - Remove "Try LanceDB Cloud" in banner |
||
|
ccmao1130
|
6bdfe73604 |
fix: add reverie banner to readme (#4346)
Add Reverie banner to the top of README until event ends. Additionally, I also removed the "LanceDB Cloud" skinny banner since we're not promoting that right? Should we modify the larger LanceDB modify to not mention "Try LanceDB Cloud"? |
||
 Will JonesandClaude
|
a68c7be202 |
feat(node): accept ArrayBuffer, Blob/File, and URL as blob column inputs (#4313)
Closes #4270. ## Summary Blob v2 columns now accept more of the containers TypeScript code commonly has, both as the column value and inside `{ data }` / `{ uri }`: | Input | Handling | |---|---| | `ArrayBuffer` | `new Uint8Array(value)`, no copy | | `URL` | stored as `value.href` | | `Blob` / `File` | `await value.arrayBuffer()` in the async write paths: `createTable`, `add`, `mergeInsert` | `makeArrowTable` is synchronous, so it can't read a `Blob`. Given one, it throws an error that names the field and row and points to the async APIs or `new Uint8Array(await value.arrayBuffer())`. It does not write a null blob. ### Types `BlobInput` is now the accepted input type. It is widened to the types above and exported from the package entry point with `BlobData` and `BlobUri`, so callers can annotate rows and `tsc` catches bad inputs. The struct forms are a union that requires exactly one of `data` / `uri`. The old normalized `{ data, uri }` shape is renamed to `BlobValue` (internal). ### Implementation - `blob.ts`: `normalizeBlobInput` (sync: `ArrayBuffer`/`URL`), `resolveBlobInput` (async: `Blob`/`File`). `coerceBlobValue` normalizes first. - `arrow.ts`: `mapBlobInputs` walks only the blob positions in the schema, including blobs nested in structs and lists. It copies records and arrays only where a value changed, so caller data is never mutated. `resolveBlobInputs` runs before `makeArrowTable` in `convertToTable` (used by `add` / `mergeInsert`) and in `createTable`. Each distinct `Blob` is read once. - `makeArrowTable` now coerces every blob value to the full `{ data, uri }` struct **before** schema inference. This also fixes two bugs that existed before this change: - Rows mixing plain bytes with `{ data }` failed inference. - A `{ uri }`-only struct produced a blob struct missing its `data` child. Blob validation errors now include the field path and row. - Also fixed along the way: - `add` / `mergeInsert` on a table with an embedding function rejected most blob values, because `ensureNestedFieldsExist` filled blob columns in like ordinary structs. It now skips them. - With a schema, a struct field that is `null` in one row and a record in a later row failed inference. Primitives and arrays given for a struct field are now rejected instead of being written as an all-null struct. - Blob structs with keys other than `data` / `uri` are rejected, which catches typos. Class instances with `data` / `uri` are handled like plain objects. ## Testing - Unit tests in `__test__/blob.test.ts`: coercion of the new types, rejection and error messages for `Blob` in the sync path, nested lists and structs, `resolveBlobInputs` (only blob columns touched, caller data untouched, shared `Blob` read once). - Table tests in `__test__/table.test.ts`: - `createTable`, `add`, and `mergeInsert` round-trip `ArrayBuffer` / `Blob` / `File` bytes through `fetchBlobs`. - `URL` and `{ uri: URL }` are added to the external-blob round-trip test from #4317 (`allowExternalBlobOutsideBases`). - Full `jest` suite passes locally after the rebase (938 passed). `pnpm lint` is clean and `pnpm run docs` has been regenerated. Streaming inputs (`ReadableStream`, Node `Readable`, async iterables) remain out of scope, as the issue says. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_012U4KdHofT5edv3LcXHXbLv --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Will JonesandClaude Opus 5.5
|
11b679a2db |
ci: add critical fixes section to release notes (#4361)
Adds a "🚨 Critical Fixes" section to the release notes, directly below Highlights, for PRs labeled `critical-fix`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
 Aviral Saxena
|
5f0bb98e90 |
fix(python): use exact integer math for list_versions timestamps (#4426)
Refs #4404. AsyncTable.list_versions split the nanosecond timestamp with float division/modulo (`ts // 1e9`, `ts % 1e9`). At ~1.8e18 a float can't hold every integer, so the microseconds drifted (.274000 came back as .273999). Use `divmod(ts, 1_000_000_000)` and integer // 1000 instead. Only the precision fix is included here. Timezone-aware UTC (option 1 in #4404) is on hold for now, and I can add it here or in a follow-up depending on what they prefer. Added test_list_versions_timestamp_precision using the value from the issue (1790989429274000000), expecting microsecond == 274000. |
||
|
Xuanwo
|
61a285f590 |
fix: preserve Function dependency errors in column operations (#4434)
Function column declarations and refreshes can return HTTP 404 when the referenced Function has been deleted. The SDK currently translates those responses into `TableNotFound`, hiding the server diagnostic and making an existing table appear missing. Preserve the HTTP error, status, and request ID for these two operations while retaining schema-cache invalidation. Ordinary table operations keep their existing `TableNotFound` behavior. |
||
|
lancedb-gatefixer[bot]
|
7f5933594f |
fix(python): reject empty query vectors before column inference (#4415)
Empty list and NumPy vector queries were validated too late: local search inferred a vector column first, producing misleading schema errors, while remote search indexed `vector[0]` and raised `IndexError`. Add a shared empty-vector check before column inference and query creation, and validate direct vector-builder inputs. Local and remote sync search now raise `ValueError: Query vector must not be empty` for these inputs. Validation: - All 26 new regression cases failed on the original code and pass with the fix, covering list/NumPy inputs, automatic/explicit vector search, inferred/explicit column names, one/two local vector columns, and remote search before query submission. - After merging current `main`, all 32 focused cases passed, covering these regressions and upstream async searches without a query. - The query, table, remote, utility, hybrid-query, and database suites passed in the rebuilt uv environment: 541 passed, 1 skipped, 1 slow test deselected. - `ruff check .`, `ruff format .`, and `cargo fmt --all` passed from the repository root. Fixes #4389 <!-- lance-gatekeeper-fix:v1 agent=f1f4875090ef391bb1aa912dd54026dc generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
3e0a1a8780 |
fix: reject zero vector search probes across SDKs (#4418)
`nprobes(0)` bypassed the validation used by the minimum and maximum probe setters, allowing zero into query requests and producing inconsistent local and remote results. Sync Python could additionally interpret the zero maximum as an unbounded search. Reject zero in the Rust vector-query builder and propagate its input error through the Python vector/hybrid and Node bindings. Validate nonpositive counts in the sync Python vector and hybrid builders so every SDK and backend reports `nprobes must be greater than 0` before execution. Update the Rust examples and document the positive-count requirement. Regression tests cover local and remote vector/hybrid builders, rejection before remote query requests, promised TypeScript query vectors, and positive counts replacing both probe bounds. Validation: - Rust probe regression tests: 2 passed. - Focused Python query, remote, hybrid, and serialization tests: 22 passed. The four sync regression cases fail with the original setters. - Focused TypeScript indexed local and mocked remote tests: 2 passed. - Workspace `cargo check --features remote --tests --examples` and Clippy passed. - Rust formatting, Python formatting/lint, and TypeScript build/lint/docs passed. Repository-wide `ruff check .` reports 29 existing errors in unchanged CI scripts; `ruff check python` passes. Fixes #4388 BREAKING CHANGE: Rust `VectorQuery::nprobes` now returns `Result<Self>` instead of `Self`; callers must handle the result, as with the existing minimum/maximum probe setters. <!-- lance-gatekeeper-fix:v1 agent=e92fda5a212c7f9e80fa55e7b4e4634d generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
 Chi-Wei LeeandWill Jones
|
0be3ae960e |
fix(python): support exist_ok in local create_table (#4421)
Synchronous local Python `create_table` callers using `mode="exist_ok"` received `ValueError` instead of creating or opening a table. Related to [#2900](https://github.com/lancedb/lancedb/issues/2900). The local wrapper rejected this mode before the shared Rust parser could validate it. Removing that guard delegates mode validation to the parser, and the sync and async docs now describe `exist_ok`. Remote option gaps and the TypeScript table type gap remain outside this change. The added tests retain both calls from [@wjones127's comment](https://github.com/lancedb/lancedb/issues/2900#issuecomment-5964167206). This builds on the code @wjones127 posted there. Three regression tests fail before the change, pass after it, fail with the original `db.py` restored, then pass again on HEAD. Tested on Linux x86_64 with Python 3.14.7. From the Python directory, `uv run --no-sync --extra tests --extra dev python -m pytest python/tests/test_db.py -q` reports 53 passed. `uv run --no-sync --extra tests --extra dev python -m pytest python/tests -q -rs -m "not slow and not s3_test"` reports 1428 passed, 34 skipped, 68 deselected, and one failure: the existing `test_get_lsm_write_spec`, also failing on clean base (1425 passed, one failed). Ruff 0.9.9 `ruff format --check .` and `ruff check .` pass. Tests used worktree Python code with the released `lancedb==0.40.0b12` native wheel. No native source build was run. Co-authored-by: Will Jones <willjones127@gmail.com> |
||
|
lancedb-gatefixer[bot]
|
9c3e1cbae7 |
fix: reject cloning onto an existing local table (#4416)
Local `clone_table` passed existing targets to Lance's clone transaction, which entered the manifest-update path and failed with `Clone operation should not enter build_manifest`. Check the target dataset using the connection's storage options and session before cloning, and return `TableAlreadyExists` when it exists. Propagate other read errors. Add regression coverage for cloning onto the source itself and onto a different existing table, with both manifest layouts, verifying that rows and version histories remain unchanged. Validation: - All four regression cases reproduced the reported internal error before the fix. - `cargo test --locked --quiet --profile ci --features remote -p lancedb --lib test_clone -j 12`: 24 passed. - `cargo fmt --all -- --check` and `git diff --check` passed. - `cargo clippy --locked --quiet --profile ci --features remote -p lancedb --tests --examples -j 12` passed. Fixes #4387 <!-- lance-gatekeeper-fix:v1 agent=522c80785eeaaef146e8179c848b903e generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
de19ecbf63 |
fix: reject empty column alterations and unknown Python keys (#4417)
Fixes #4390 Lance accepts column alterations with every change field unset, so a path-only Python call could commit a new local version without changing the schema. The Python binding also silently discarded unknown keys such as `nulable`. Validate every alteration in the shared Rust `Table` API before dispatching to either backend, and reject unknown dictionary keys in the Python binding while preserving the legacy `name` alias. Invalid batches leave local schema and version unchanged and send no remote request. Validation: - Reproduced both reported calls on the original extension: versions advanced from 1 to 2 to 3 with an unchanged schema; all 13 new invalid-input Python cases failed before the fix. - Rebuilt the extension with `uv run --extra tests --extra dev maturin develop --extras tests,dev`; all 18 focused Python cases pass across sync/async and local/mocked remote tables. - `cargo test --quiet --features remote -p lancedb --lib alter_column`: all eight tests pass, including both new regressions. - `cargo clippy --quiet --features remote --tests --examples` passes across the workspace. - `cargo fmt --all`, `ruff check .`, and `ruff format .` pass. - `typos` 1.26.0 passes across the repository; intentional misspellings in regression fixtures use the existing line-specific spelling exemptions. <!-- lance-gatekeeper-fix:v1 agent=36a547cd11108898893456042e38628b generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
57a952135b |
fix(python): support async search without a query (#4419)
`AsyncTable.search()` and `search(None)` reached vector search without assigning `vector_query`, raising `UnboundLocalError` on both local and remote tables. Return a plain `AsyncQuery` before inferring the search type, equivalent to `table.query()`, and update the overloads and documentation to describe that return value. Add six regression cases for omitted and explicit `None` queries on local tables with and without vectors and on a mocked remote endpoint. They verify scans, limits, filters, column projection, and the remote request body. Validation: all six regressions reproduced the original `UnboundLocalError`; all 11 focused query tests pass with the fix, including existing vector, full-text, hybrid, and sync scan coverage. The uv development bootstrap, locked Ruff formatting/lint checks, and focused Pyright return-type assertions also pass. Fixes #4385 <!-- lance-gatekeeper-fix:v1 agent=bb9a7509f6dadde030d8113b172abc9a generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
6851133a7c |
fix(python): serialize deprecated v2 manifest option (#4420)
Local synchronous `create_table(..., enable_v2_manifest_paths=True/False)` failed with `TypeError` because the deprecated argument was forwarded as a boolean in `storage_options`, while the Rust binding expects string values. Convert the argument to lowercase `"true"` or `"false"` before forwarding it, preserving the deprecation warning. Add regression coverage for both values with populated and schema-only tables, checking stored data and the manifest mode after reopening. Validation: - All four regression cases reproduced the original `TypeError` before the fix and pass afterward. - `uv run --no-sync --extra tests --extra dev pytest python/tests/test_db.py -q --tb=short -m 'not slow and not s3_test'`: 54 passed. - Repository-wide `ruff format .` and `ruff check .` completed successfully. Fixes #4384 <!-- lance-gatekeeper-fix:v1 agent=9c14955b90d0c12d6a46605ec5ef32c0 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
8c2c39b3c4 |
fix(python): repair sync manifest path migration (#4422)
Fixes #4383 `LanceTable.migrate_v2_manifest_paths()` raised `AttributeError` because it called a nonexistent `AsyncTable.migrate_v2_manifest_paths()` method. Delegate to the existing async method, `migrate_manifest_paths_v2()`, while preserving the public sync API. Add a local regression test for tables starting with either v1 or v2 manifest paths. It checks repeated migration, v2 manifest filenames on disk, and preservation of data and version history after reopening the table. Validation: - Both new regression cases failed with the reported `AttributeError` before the fix. - `cd python && uv run --extra tests pytest python/tests/test_table.py::test_migrate_v2_manifest_paths python/tests/test_db.py::test_create_table_v2_manifest_paths_async -q` — 3 passed. - Repository-wide `ruff check .` and `ruff format .` passed using Ruff 0.15.20 from `python/uv.lock`. - `git diff --check` passed. <!-- lance-gatekeeper-fix:v1 agent=bb500f0d1ddefeba3c7d7d8390a2fb74 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
7e979c1416 |
fix(node): preserve schemas for empty query results (#4423)
Empty local TypeScript queries returned a zero-row Arrow table with no columns because `toArrow()` built the table solely from record batches. Local streams can yield no batches, while remote streams often supply an empty batch carrying the schema. Expose the executed stream's schema through the native iterator and decode it into an empty Arrow table when no batches arrive. This preserves projected and search columns, types, nullability, and metadata without executing another query. Document the empty-result behavior. Validation: - Eleven regression cases failed with the original `toArrow()` implementation; all twelve now pass, including remote schema-only and empty-batch responses. - All 1,075 tests in the query, remote, Arrow, and table suites pass. - Native build, TypeScript compilation, Rust formatting and Clippy (`-D warnings`), TypeScript lint/formatting, and documentation generation pass. Fixes #4386 <!-- lance-gatekeeper-fix:v1 agent=7d589da1f7db5c7383f9600490e111a9 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
cc6b4aa4b4 |
fix(remote): preserve schema for empty merge insert sources (#4425)
Remote `merge_insert` buffered its reader into a `Vec<RecordBatch>`, losing the schema when the reader yielded no batches. Scanning that vector failed before the request could reach the server. Preserve the reader's schema in an empty record batch when buffering produces no batches. The source remains replayable for retries, and empty-source merges still reach the server so `when_not_matched_by_source_delete` can apply deletes. The regression tests decode the outgoing Arrow IPC and verify the source schema, zero rows, merge options, and result statistics on an initial request and a retry after HTTP 409. They cover readers with no batches and readers with an empty batch, in insert and delete modes, including an ID-only delete source. Validation: - Before the fix, both no-batch cases reproduced `Cannot scan an empty Vec<RecordBatch>`; the empty-batch cases passed. - `cargo test --quiet --profile ci --features remote -p lancedb --lib remote::table::tests::test_merge_insert` — 9 passed. - `cargo fmt --all` and `cargo fmt --all -- --check`. - `cargo clippy --quiet --profile ci --features remote -p lancedb --tests --examples`. Fixes #4380 <!-- lance-gatekeeper-fix:v1 agent=a48c9fc83772479990d76124f3a2b3be generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
3d8c9e2ddc |
fix(python): close synchronous database connections (#4424)
Calling `close()` on a synchronous remote connection returned an unawaited coroutine and left the connection usable. Local synchronous connections had no close or context-manager API. Add synchronous `close()` and `is_open()` to the connection interface, delegate them to the existing connection handle for local, remote, and namespace connections, and provide shared context-manager cleanup. Closing is idempotent, and subsequent database operations use the existing `Connection is closed` error. Document the lifecycle with examples. Validation: - Reproduced the reported coroutine warning and missing local lifecycle API on the original code; all 15 new regression cases failed before the fix. - All 168 tests in `test_db.py`, `test_remote_db.py`, and `test_namespace.py` passed through the uv-managed environment, including local, in-memory, namespace, native-derived, and mocked remote lifecycle coverage. - All 6 synchronous connection doctests passed. - Repository-wide Ruff 0.9.9 lint and formatting passed; `git diff --check` passed. Fixes #4382 <!-- lance-gatekeeper-fix:v1 agent=a743139af98198f1d4c8934c9cc1d0c9 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
Will Jones
|
b0c0d8ee92 | perf(remote): size blob file handles from one descriptor take (#4349) | ||
 
|
2734e10767 |
fix(node): update arrow to 21 (#4362)
Thanks @vincentkoc for starting this! I was just looking into this myself. I started with your PR and added: - make arrow 21 the default - widen the peer range to <22 so if they release 21.3 that'll be included - have to do skipLibCheck in tsconfig.json, because of flatbuffers 25 which needs TypeScript 5.7 - fix override in Symbol.hasInstance - support new types in Arrow 20-21 (LargeList, Utf8View, BinaryView, IntervalMonthDayNano) - arrow-compat job in nodejs.yml that runs tests with arrow 19 and 20 (adds 2 sec) - put the list of arrow types in one place - update NODEJS_THIRD_PARTY_LICENSES.md - regenerated typescript docs (sorry about reorderings, one time cost) - reformat arrow.test.ts (sorry, almost all whitespace, thanks to biome. git diff -w shows 110 lines edited) - fixed Sliced Utf8View / BinaryView serialization error, by just converting them to Utf8 and Binary (as our rust code would anyway) --------- Co-authored-by: Vincent Koc <vincentkoc@ieee.org> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
 dependabot[bot]
|
f72779ebb5 |
build(deps): bump the rust-minor-patch group across 1 directory with 7 updates (#4355)
Bumps the rust-minor-patch group with 7 updates in the / directory: | Package | From | To | | --- | --- | --- | | [aws-smithy-types](https://github.com/smithy-lang/smithy-rs) | `1.6.3` | `1.6.4` | | [object_store](https://github.com/apache/arrow-rs-object-store) | `0.14.1` | `0.14.2` | | [serde_with](https://github.com/jonasbb/serde_with) | `3.23.0` | `3.24.0` | | [napi](https://github.com/napi-rs/napi-rs) | `3.12.2` | `3.12.5` | | [napi-derive](https://github.com/napi-rs/napi-rs) | `3.6.5` | `3.6.9` | | [napi-build](https://github.com/napi-rs/napi-rs) | `2.4.2` | `2.5.0` | | [arc-swap](https://github.com/vorner/arc-swap) | `1.9.1` | `1.9.2` | Updates `aws-smithy-types` from 1.6.3 to 1.6.4 <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/smithy-lang/smithy-rs/commits">compare view</a></li> </ul> </details> <br /> Updates `object_store` from 0.14.1 to 0.14.2 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/apache/arrow-rs-object-store/releases">object_store's releases</a>.</em></p> <blockquote> <h2>object_store v0.14.2</h2> <!-- raw HTML omitted --> <h1>Changelog</h1> <h2><a href="https://github.com/apache/arrow-rs-object-store/tree/v0.14.2">v0.14.2</a> (2026-09-11)</h2> <p><a href="https://github.com/apache/arrow-rs-object-store/compare/v0.14.1...v0.14.2">Full Changelog</a></p> <p><strong>Implemented enhancements:</strong></p> <ul> <li>Implement <code>Clone</code> for <code>MicrosoftAzure</code> <a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/824">#824</a></li> <li>Support URLs for jurisdiction-restricted buckets on R2 <a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/814">#814</a></li> <li>Lightweight DNS cache with shuffle to prevent DNS flooding <a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/726">#726</a></li> <li>Allow presigning object_store multipart uploads <a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/271">#271</a></li> <li>Allow specifying <code>Content-Type</code> when signing object_store PUT requests <a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/270">#270</a></li> </ul> <p><strong>Fixed bugs:</strong></p> <ul> <li>Incomplete ASF license header in <code>src/client/s3.rs</code> (caused -1 vote on 0.14.2 RC1) <a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/853">#853</a></li> <li>WriteMultipart::finish should abort after part upload failure <a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/818">#818</a></li> <li>The 1.85 MSRV is inaccurate with all features <a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/811">#811</a></li> <li>When the backend service returns an HTTP 500, the object store panics <a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/414">#414</a></li> </ul> <p><strong>Documentation updates:</strong></p> <ul> <li>Docs: point users uploading large objects at the multipart API <a href="https://redirect.github.com/apache/arrow-rs-object-store/pull/839">#839</a> (<a href="https://github.com/alamb">alamb</a>)</li> <li>[object-store]: update release schedule <a href="https://redirect.github.com/apache/arrow-rs-object-store/pull/834">#834</a> (<a href="https://github.com/alamb">alamb</a>)</li> <li>Add doc example for multipart upload to <code>GoogleCloudStorage::create_multipart</code> <a href="https://redirect.github.com/apache/arrow-rs-object-store/pull/803">#803</a> (<a href="https://github.com/alamb">alamb</a>)</li> <li>Add doc example for multipart upload to <code>MicrosoftAzure::create_multipart</code> <a href="https://redirect.github.com/apache/arrow-rs-object-store/pull/802">#802</a> (<a href="https://github.com/alamb">alamb</a>)</li> <li>Add doc example for multipart upload to AmazonS3::create_multipart <a href="https://redirect.github.com/apache/arrow-rs-object-store/pull/801">#801</a> (<a href="https://github.com/alamb">alamb</a>)</li> </ul> <p><strong>Closed issues:</strong></p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/apache/arrow-rs-object-store/blob/main/CHANGELOG-old.md">object_store's changelog</a>.</em></p> <blockquote> <!-- raw HTML omitted --> <h1>Historical Changelog</h1> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/arrow-rs-object-store/commit/279572ea60f3a7a6e5237e066a6f4a35eee611e0"><code>279572e</code></a> Update CHANGELOG for latest main (<a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/855">#855</a>)</li> <li><a href="https://github.com/apache/arrow-rs-object-store/commit/4ff7d048073c07bcc1f1dca74436245b2ba20dda"><code>4ff7d04</code></a> Restore missing first line of ASF license header in src/client/s3.rs (<a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/854">#854</a>)</li> <li><a href="https://github.com/apache/arrow-rs-object-store/commit/25fa079c41fecb8011d5b8c3019f20e09c9b0c74"><code>25fa079</code></a> Update version to 0.14.2 and add CHANGELOG (<a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/850">#850</a>)</li> <li><a href="https://github.com/apache/arrow-rs-object-store/commit/c37db72b92b78608291c1b33feb79bb5bb6d2d1c"><code>c37db72</code></a> feat: retry failed multipart part uploads (<a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/849">#849</a>)</li> <li><a href="https://github.com/apache/arrow-rs-object-store/commit/135984fdf91f813c48c47b6288586d9b0e205164"><code>135984f</code></a> Add doc hints pointing large-object uploads at the multipart API (<a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/839">#839</a>)</li> <li><a href="https://github.com/apache/arrow-rs-object-store/commit/b07471e2bc341278f86e30cf80a850d56cbe2c67"><code>b07471e</code></a> [object-store]: update release schedule (<a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/834">#834</a>)</li> <li><a href="https://github.com/apache/arrow-rs-object-store/commit/978fb48b9ea6ebe3f8f4833f656e793cde79ab15"><code>978fb48</code></a> Fix MSRV violation in <code>parse.rs</code> test, check <code>--all-targets</code> in MSRV CI (<a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/837">#837</a>)</li> <li><a href="https://github.com/apache/arrow-rs-object-store/commit/e2a49349622aa7ad5db12b107a557d957ac01241"><code>e2a4934</code></a> Speed up MSRV CI check by installing cargo-msrv via taiki-e/install-action (#...</li> <li><a href="https://github.com/apache/arrow-rs-object-store/commit/e28807de7338abebf14ecf2ab959294cb3096d83"><code>e28807d</code></a> feat: presigned URLs with extra query params and signed headers (SignedUrlOpt...</li> <li><a href="https://github.com/apache/arrow-rs-object-store/commit/5c743637e97d2f3417b6fd54532bb1223c6547c1"><code>5c74363</code></a> Restore UnwindSafe/RefUnwindSafe on ClientOptions by bounding DnsResolver (<a href="https://redirect.github.com/apache/arrow-rs-object-store/issues/836">#836</a>)</li> <li>Additional commits viewable in <a href="https://github.com/apache/arrow-rs-object-store/compare/v0.14.1...v0.14.2">compare view</a></li> </ul> </details> <br /> Updates `serde_with` from 3.23.0 to 3.24.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/jonasbb/serde_with/releases">serde_with's releases</a>.</em></p> <blockquote> <h2>serde_with v3.24.0</h2> <h3>Added</h3> <ul> <li>Implement <code>JsonSchemaAs</code> for <code>NoneAsEmptyString</code>, so fields using it compile with the <code>schemars_*</code> features (<a href="https://redirect.github.com/jonasbb/serde_with/issues/1014">#1014</a>)</li> </ul> <h3>Changed</h3> <ul> <li>Make <code>with_prefix!</code> and <code>with_suffix!</code> available with the <code>alloc</code> feature, instead of only with <code>std</code> (<a href="https://redirect.github.com/jonasbb/serde_with/issues/1008">#1008</a>)</li> </ul> <h3>Fixed</h3> <ul> <li>Handle potential panics in deserialization of chrono and time DateTimes. This was privately reported by <a href="https://github.com/NotAFlightRisk"><code>@NotAFlightRisk</code></a>.</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/jonasbb/serde_with/commit/35ac84c8faf34d6822991bbf2397056994c9208b"><code>35ac84c</code></a> Bump version to 3.24 (<a href="https://redirect.github.com/jonasbb/serde_with/issues/1016">#1016</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/7b8e561c709544b417b3c0f05f3c57a753e86417"><code>7b8e561</code></a> Bump version to 3.24</li> <li><a href="https://github.com/jonasbb/serde_with/commit/6e576c8d5cc5f31efd26cf783d76eac6607d784e"><code>6e576c8</code></a> Bump the github-actions group with 2 updates (<a href="https://redirect.github.com/jonasbb/serde_with/issues/1012">#1012</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/7b1814b5cf8f2ce4ef7844105ab26760371af0cf"><code>7b1814b</code></a> Add schemars support for NoneAsEmptyString (<a href="https://redirect.github.com/jonasbb/serde_with/issues/1014">#1014</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/92eeedd1e5be0f176bd9a0532eb8ab1503396bb2"><code>92eeedd</code></a> Handle potential panic during deserialization for large datetimes (<a href="https://redirect.github.com/jonasbb/serde_with/issues/1015">#1015</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/3404642aae13a3dfcf4ea0f47749f759b36a85db"><code>3404642</code></a> Handle potential panic during deserialization for large datetimes</li> <li><a href="https://github.com/jonasbb/serde_with/commit/c7073b98f150d60c22e1ccdb49f1c7151dd8595f"><code>c7073b9</code></a> Add schemars support for NoneAsEmptyString</li> <li><a href="https://github.com/jonasbb/serde_with/commit/6b32290267fa18064894e2fa67bf4bed7fb4bd5f"><code>6b32290</code></a> Run test with only the alloc feature (<a href="https://redirect.github.com/jonasbb/serde_with/issues/1013">#1013</a>)</li> <li><a href="https://github.com/jonasbb/serde_with/commit/3361c69817d02b079b15f71d1281f573594cc948"><code>3361c69</code></a> Run test with only the alloc feature</li> <li><a href="https://github.com/jonasbb/serde_with/commit/b43bb81cd8f02f06c2bf4bcd057c83e71442a515"><code>b43bb81</code></a> Reduce required feature for with_{prefix,suffix} from std to alloc (<a href="https://redirect.github.com/jonasbb/serde_with/issues/1008">#1008</a>)</li> <li>Additional commits viewable in <a href="https://github.com/jonasbb/serde_with/compare/v3.23.0...v3.24.0">compare view</a></li> </ul> </details> <br /> Updates `napi` from 3.12.2 to 3.12.5 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/napi-rs/napi-rs/releases">napi's releases</a>.</em></p> <blockquote> <h2>napi-v3.12.5</h2> <h3>Fixed</h3> <ul> <li>fatal_exception noop in optimized builds (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3509">#3509</a>)</li> </ul> <h2>napi-v3.12.4</h2> <h3>Security</h3> <ul> <li><code>Object::unwrap</code> and <code>Object::remove_wrapped</code> (and the deprecated <code>Env::unwrap</code> / <code>Env::drop_wrapped</code>) read a <code>TypeId</code> out of a payload that is not a <code>TaggedObject</code>, giving a heap out-of-bounds read and a JavaScript-forgeable out-of-bounds write; payload provenance is now validated before dereferencing. See <a href="https://github.com/napi-rs/napi-rs/security/advisories/GHSA-rhpj-pggq-896v">GHSA-rhpj-pggq-896v</a>, fixed in <a href="https://redirect.github.com/napi-rs/napi-rs/pull/3506">#3506</a>.</li> <li>The serde deserializer reached <code>unreachable!()</code> on ordinary JavaScript values and aborted the process instead of returning an error. See <a href="https://github.com/napi-rs/napi-rs/security/advisories/GHSA-f334-75xc-qxv3">GHSA-f334-75xc-qxv3</a>, fixed in <a href="https://redirect.github.com/napi-rs/napi-rs/pull/3507">#3507</a>.</li> <li><code>ArrayBuffer::from_external</code>, <code>TypedArraySlice::from_external</code> and <code>Uint8ClampedSlice::from_external</code> (plus the compat-mode <code>Env::create_arraybuffer_with_borrowed_data</code>) built their slice from freed memory on runtimes without external buffers. See <a href="https://github.com/napi-rs/napi-rs/security/advisories/GHSA-32mm-r9wp-hrvc">GHSA-32mm-r9wp-hrvc</a>, fixed in <a href="https://redirect.github.com/napi-rs/napi-rs/pull/3503">#3503</a>.</li> </ul> <h3>Fixed</h3> <ul> <li><em>(napi)</em> return errors from the serde deserializer for unexpected JS value shapes (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3507">#3507</a>)</li> <li><em>(napi)</em> validate wrapped payload provenance in Object::unwrap/remove_wrapped (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3506">#3506</a>)</li> <li><em>(napi)</em> point from_external slices at the engine-owned copy after finalize reclaims the source (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3503">#3503</a>)</li> </ul> <h2>napi-v3.12.3</h2> <h3>Fixed</h3> <ul> <li>CurrentThread waker-stack deadlocks and threadless-wasm Buffer detachment (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3489">#3489</a>)</li> <li><em>(napi)</em> catch panics in closure trampolines instead of aborting (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3473">#3473</a>)</li> </ul> <h3>Other</h3> <ul> <li><em>(napi)</em> keep anyhow error cause when converting (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3477">#3477</a>)</li> <li><em>(napi)</em> outline threadsafe callback dispatch (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3465">#3465</a>)</li> <li><em>(napi)</em> outline class instance construction (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3464">#3464</a>)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/napi-rs/napi-rs/commit/43100baf28a3e5709641e35f892be4da5d62dcb2"><code>43100ba</code></a> chore: release (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3508">#3508</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/bb7b75a7d53b0757f99813c7fef0537cef044adf"><code>bb7b75a</code></a> chore(release): publish</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/89d7a0b3636b4454265363dab58df471030b4f85"><code>89d7a0b</code></a> chore(deps): update dependency oxfmt to ^0.68.0 (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3519">#3519</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/55545910c2487da38453e6171330e8fc9133213c"><code>5554591</code></a> fix(cli): rate-limit failed incarnation probes and let reconciliation tests i...</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/6d10d826c34f5d0009eb1ab47efaa4a2e34f4555"><code>6d10d82</code></a> feat(cli): make the deferred workerd loader usable by host-driven addons (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3520">#3520</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/a1bcecbd1bef2f51f1047f5800d5a312eb2eba3b"><code>a1bcecb</code></a> feat(cli): add <code>napi.wasm.threadlessInitialMemory</code> for threadless WASI loader...</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/0999babc1e37ce869b79f59db14f9eb12f6647df"><code>0999bab</code></a> feat(cli): export <code>__napiBindingTarget</code> from generated binding loaders (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3515">#3515</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/30ea504c3d65e1cb178d3124068257d8df8f3835"><code>30ea504</code></a> fix(cli): bound symlink-ancestor reconciliation guards to the anchor's parent...</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/ae8f34a8dc5f629b53d136442e8f614f2d3c17b6"><code>ae8f34a</code></a> fix(cli): run the wasm env cleanup barrier on a raw emnapi `Context.destroy()...</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/ccd314bbb5201182311b74dddd61f60b323318f4"><code>ccd314b</code></a> fix(cli): never wedge reconciliation locks when host identity probes fail (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3">#3</a>...</li> <li>Additional commits viewable in <a href="https://github.com/napi-rs/napi-rs/compare/napi-v3.12.2...napi-v3.12.5">compare view</a></li> </ul> </details> <br /> Updates `napi-derive` from 3.6.5 to 3.6.9 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/napi-rs/napi-rs/releases">napi-derive's releases</a>.</em></p> <blockquote> <h2>napi-derive-v3.6.9</h2> <h3>Other</h3> <ul> <li>update Cargo.toml dependencies</li> </ul> <h2>napi-derive-v3.6.8</h2> <h3>Fixed</h3> <ul> <li><em>(napi)</em> guard AsyncTask completion against env teardown (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3536">#3536</a>)</li> </ul> <h2>napi-derive-v3.6.7</h2> <h3>Other</h3> <ul> <li>update Cargo.toml dependencies</li> </ul> <h2>napi-derive-v3.6.6</h2> <h3>Other</h3> <ul> <li>update Cargo.toml dependencies</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/napi-rs/napi-rs/commit/2763e12efc855748485129952a6ccb97ac991c06"><code>2763e12</code></a> chore: release (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3538">#3538</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/5ae5cf604809ed6b9868c8e67b201bc9eb68d9bf"><code>5ae5cf6</code></a> chore(release): publish</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/0da138ec836906da5bc3a996521299e1d284bdda"><code>0da138e</code></a> feat(cli): self-sign OpenHarmony artifacts with --ohos-sign (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3539">#3539</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/2499a6311148492bc01ae316621b514a9345e4e7"><code>2499a63</code></a> feat(async-runtime): offer the MultiThread flavor on wasm32-wasip1-threads (#...</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/886bd071972e09010ee901d1a1bb669a26d3e0dc"><code>886bd07</code></a> chore(deps): update dependency oxc-parser to ^0.151.0 (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3542">#3542</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/edb0a295a14371b2f943c2a89bacf385eb61a038"><code>edb0a29</code></a> fix(napi): validate native payload provenance for External and instance data ...</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/38162bb0eb324ae24b402982bad3d4ef3f24c90a"><code>38162bb</code></a> chore: release (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3529">#3529</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/ec3d8ead66c18e022d2e3fb6832e52e5204ffcc3"><code>ec3d8ea</code></a> fix(napi): guard AsyncTask completion against env teardown (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3536">#3536</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/96ed2672820b5468b6a451080fc27e6eef594d3d"><code>96ed267</code></a> chore(deps): update release-plz/action action to v0.5.139 (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3537">#3537</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/b2c9f3b610cb167d2c05b8d6c748c120926ffd73"><code>b2c9f3b</code></a> chore(deps): update release-plz/action action to v0.5.138 (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3533">#3533</a>)</li> <li>Additional commits viewable in <a href="https://github.com/napi-rs/napi-rs/compare/napi-derive-v3.6.5...napi-derive-v3.6.9">compare view</a></li> </ul> </details> <br /> Updates `napi-build` from 2.4.2 to 2.5.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/napi-rs/napi-rs/releases">napi-build's releases</a>.</em></p> <blockquote> <h2>napi-build-v2.5.0</h2> <h3>Added</h3> <ul> <li><em>(async-runtime)</em> offer the MultiThread flavor on wasm32-wasip1-threads (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3541">#3541</a>)</li> </ul> <h2>napi-build-v2.4.4</h2> <h3>Fixed</h3> <ul> <li><em>(napi)</em> guard AsyncTask completion against env teardown (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3536">#3536</a>)</li> </ul> <h2>napi-build-v2.4.3</h2> <h3>Fixed</h3> <ul> <li><em>(cli)</em> drain outstanding async work before disposing a WASI binding (<a href="https://redirect.github.com/napi-rs/napi-rs/pull/3528">#3528</a>)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/napi-rs/napi-rs/commit/2763e12efc855748485129952a6ccb97ac991c06"><code>2763e12</code></a> chore: release (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3538">#3538</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/5ae5cf604809ed6b9868c8e67b201bc9eb68d9bf"><code>5ae5cf6</code></a> chore(release): publish</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/0da138ec836906da5bc3a996521299e1d284bdda"><code>0da138e</code></a> feat(cli): self-sign OpenHarmony artifacts with --ohos-sign (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3539">#3539</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/2499a6311148492bc01ae316621b514a9345e4e7"><code>2499a63</code></a> feat(async-runtime): offer the MultiThread flavor on wasm32-wasip1-threads (#...</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/886bd071972e09010ee901d1a1bb669a26d3e0dc"><code>886bd07</code></a> chore(deps): update dependency oxc-parser to ^0.151.0 (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3542">#3542</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/edb0a295a14371b2f943c2a89bacf385eb61a038"><code>edb0a29</code></a> fix(napi): validate native payload provenance for External and instance data ...</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/38162bb0eb324ae24b402982bad3d4ef3f24c90a"><code>38162bb</code></a> chore: release (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3529">#3529</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/ec3d8ead66c18e022d2e3fb6832e52e5204ffcc3"><code>ec3d8ea</code></a> fix(napi): guard AsyncTask completion against env teardown (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3536">#3536</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/96ed2672820b5468b6a451080fc27e6eef594d3d"><code>96ed267</code></a> chore(deps): update release-plz/action action to v0.5.139 (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3537">#3537</a>)</li> <li><a href="https://github.com/napi-rs/napi-rs/commit/b2c9f3b610cb167d2c05b8d6c748c120926ffd73"><code>b2c9f3b</code></a> chore(deps): update release-plz/action action to v0.5.138 (<a href="https://redirect.github.com/napi-rs/napi-rs/issues/3533">#3533</a>)</li> <li>Additional commits viewable in <a href="https://github.com/napi-rs/napi-rs/compare/napi-build-v2.4.2...napi-build-v2.5.0">compare view</a></li> </ul> </details> <br /> Updates `arc-swap` from 1.9.1 to 1.9.2 <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/vorner/arc-swap/blob/master/CHANGELOG.md">arc-swap's changelog</a>.</em></p> <blockquote> <h1>1.9.2</h1> <ul> <li>Document RefCnt must not panic (<a href="https://redirect.github.com/vorner/arc-swap/issues/208">#208</a>).</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/vorner/arc-swap/commits">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
XY Zhan
|
723a2394e5 |
feat: maintain every index a table has, including ones created later (#4342)
**The maintained set is an intent, not a snapshot.** An unnamed set is stored as the intent to maintain every index, which lance re-resolves whenever it builds a MemTable, so an index created after the spec installs is maintained too. A named set is checked here, where the caller can be told which names exist; an index of a kind the MemWAL cannot mirror fails a named set and is skipped by an unnamed one. **A newly created index reaches a writer that is already open.** lancedb caches a `ShardWriter` per shard and reuses it, so `create_index` installs the current set on the cached writer. A failure there is reported rather than propagated: the index is committed durably by then, and the writer this process holds may be fenced or poisoned, which is no reason to fail creating an index. **A table carries one spec.** `set_lsm_write_spec` fails while one is installed and names `unset_lsm_write_spec` in the error, because the generations already written were homed under that spec. Accepting a repeat that changed only the maintained set would hand a caller who also passed a different sharding a success that changed nothing they asked for. **The read path asks what is actually carried.** An unnamed set leaves the stored list empty, so the list alone cannot decide whether an index is maintained. The read asks the spec *and* every resident MemTable whether it carries the index, so a MemTable built before the index joined the set refuses the read rather than answering without its rows. Both spec routes report an unnamed set as `null` rather than `[]`: maintain whatever the table gains, and maintain nothing, are different settings. ## Known gap The guard above covers resident MemTables. Once a writer flushes and closes none is resident, so a retained generation written before the index existed still gets an indexed read that omits its rows. This window is **wider than on main**: a set resolved at install time excluded an index created later, so the read was refused; an intent includes it, and the refusal only holds while a MemTable is resident. Closing it needs the generation's own coverage, which this layer cannot see — its snapshot carries a generation and a path — so it belongs in lance's scanner or flush path. ## Testing `cargo nextest run -p lancedb --lib --features remote` — **1479 passed, 3 skipped**. `cargo fmt --all --check`, and `cargo clippy --features remote --lib --tests --examples -- -D warnings`, both clean. Covered: the round trip of an unnamed set, a set over an installed spec refused, a spec replaced through unset, a read refused while a resident MemTable predates the index, and an index created mid-write reaching the open writer. Two Python binding tests move with the semantics: an unnamed set now reads back as `None` rather than the list it resolves to, and the refusal is matched on `already set`. Neither was run locally — no Python build here — so CI is the check on them. |
||
|
857e455361 |
chore: update lance dependency to v13.1.0-beta.1 (#4358)
Updates the Rust workspace and Java Lance dependencies to [v13.1.0-beta.1](https://github.com/lance-format/lance/releases/tag/v13.1.0-beta.1) and refreshes Cargo.lock. Removes a redundant `pub(crate)` declaration in the private Node.js blob module to satisfy the required Clippy check. Validation: `cargo clippy --quiet --workspace --tests --all-features -- -D warnings`, `cargo fmt --all --quiet`, `pnpm build`, and `pnpm test __test__/blob.test.ts --runInBand` (26 tests passed) in `nodejs`. --------- Co-authored-by: Daniel Rammer <hamersaw@protonmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Lance Release
|
4730f9036b | Bump version: 0.40.0-beta.11 → 0.40.0-beta.12 | ||
 Jonathan HsiehandClaude Opus 5.5
|
82d39485a4 |
feat(functions): address Functions by namespace path (#4341)
Adds namespace addressing to Functions, matching what Secrets already have (#4150). ## API `create_function` / `create_function_async`, `get_function`, `list_functions`, `drop_function` / `drop_function_async` take `namespace_path` — keyword-only in Python (sync, async and remote connections) and a trailing `&[String]` in Rust — defaulting to the root. The namespace is addressed in the route identifier (`/v1/function/analytics$features$embed/describe`, `/v1/namespace/analytics$features/function/list`); no body carries it. `FunctionVersion` and `FunctionVersionRef` name their Function **by parts** — a short `name` plus `namespace_path` — the same shape as `SecretReference`, so a returned version reopens from its own fields: ```python fv = db.create_function(embed, namespace_path=["analytics", "features"]) fv.name, fv.namespace_path # "embed", ("analytics", "features") db.get_function(fv.name, version=fv.version, namespace_path=list(fv.namespace_path)) ``` An empty path is omitted from the wire, so root documents keep their shape. `namespace_path` is passed straight to the binding, so a bare string (`namespace_path="prod"`) is a `TypeError` rather than four one-character segments; segments are checked for addressability before a request is built. The binding and application allow-lists now share one field list, `FUNCTION_VERSION_REF_FIELDS`, which admits `namespace_path` — without it a namespaced binding or application read as coming from a newer contract. ## Service side The Sophon half serves and persists FunctionVersion / FunctionVersionRef documents in this split form, keeping the joined id as its internal catalog key. It lands in sophon with the submodule bump. ## Tests - Rust: every Function verb addresses a namespace in the path; an unaddressable segment never reaches the transport; `FunctionVersion` round-trips a namespace and `reference()` carries it; a namespaced binding and application are known shapes. - Python: sync and async verbs address the namespace; a created Function reopens from its own `name`/`namespace_path`; a bare-string path is refused before any request. Draft: compiles, `cargo fmt --check` and clippy `-D warnings` clean; lancedb lib Function tests (116) and `first_class_function_slice2` pass on this base. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |