rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-09-21 12:35:42 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,982 Commits 333 Branches 539 Tags
3fca33fcb5555175df53c7e4243be3ebab3caf62
Commit Graph
2982 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Bruno RamirezandClaude Sonnet 5 3fca33fcb5 fix: shut down the shared Tokio runtime on interpreter exit (#4175) 
Short-lived Python processes using this client can occasionally crash
with SIGABRT during interpreter shutdown, even after every operation
they ran completed successfully. The cause is the shared Tokio runtime
backing every async call: it's never told to shut down at normal process
exit, only reset (and deliberately leaked) on `fork()`. Its worker
threads keep running, uncoordinated with the interpreter, until the
process actually ends, and if one is mid-task exactly as `Py_Finalize`
starts tearing down interpreter state, it can panic on state that's
already gone. That panic happens on a background thread with no
PyO3-wrapped call frame to catch it, so Rust aborts the whole process
instead of just failing that one call. This PR gives the runtime a
coordinated, bounded shutdown by registering a Python `atexit` callback
that runs while the interpreter is still fully valid.

Getting the exit lifecycle right took a few rounds of review. Earlier
versions freed the runtime as soon as `Arc::strong_count` looked low,
but that's the wrong signal — it reflects who currently holds a
reference, not who's logically still in flight. That mistake showed up
three ways: a caller could dereference memory already freed out from
under it; an install already in progress could finish invisibly after
`shutdown()` had already decided there was nothing to do; and a spawned
task could end up as the final owner of the `Runtime`, so completing it
dropped the runtime from inside one of its own worker threads, which
Tokio itself forbids and panics on (this reproduced unprompted in this
branch's own test suite). Fixing all three meant replacing
reference-count-based tracking with an explicit counter of in-flight
top-level calls that `shutdown()` waits on directly.

This was accomplished with the following changes:
- The runtime lives in an `ArcSwapOption<Tagged>`, where `Tagged` pairs
the `Runtime` with the fork generation it was built in.
- An `OUTSTANDING` counter, incremented before a top-level
`spawn`/`spawn_blocking`/`block_on` call does anything else and
decremented only once it has truly finished (via an `OutstandingGuard`
token that carries no reference to the runtime), is what `shutdown()`
waits on — not `Arc::strong_count` or whether the slot looks empty. This
closes the install-race and makes it impossible for a task's own
completion to be the runtime's final drop.
- Once `shutdown()`'s bound elapses, it stops waiting and attempts
retirement anyway, rather than returning with the runtime and its
workers left fully alive.
- `spawn`/`spawn_blocking` use `Handle::try_current()` to pin any nested
spawn (`future_into_py` spawns a task that itself spawns a second one
for the real work) to whichever runtime is already executing it, so a
reclaim landing between the two calls can't split one logical operation
across two different runtime instances.
- The fork-child handler now only bumps a bare `GENERATION` counter — no
`ArcSwapOption` call of any kind from that context, since
`swap`/`compare_and_swap` do real reader-reconciliation work
(thread-local state, potentially an allocation) that isn't safe in a
forked child. `get_runtime()` compares its installed runtime's
generation against the live counter from ordinary context and rebuilds
on a mismatch.
- Registered `shutdown_runtime` as a Python `atexit` callback in the
`_lancedb` module init, running with the GIL released (`Python::detach`)
since the bounded wait could otherwise deadlock against any in-flight
task that itself needs the GIL.

### Testing
- Unit tests in `runtime.rs` cover: shutdown with no runtime created,
shutdown after use and lazy rebuild afterward, calling shutdown twice in
a row, a concurrent stress test racing many threads against shutdown, a
nested-spawn test reproducing `future_into_py`'s own
spawn-within-a-spawn shape under concurrent shutdown, a test confirming
a top-level task in flight survives a concurrent shutdown reclaim, and a
test forcing the install-vs-shutdown race directly.
- Built the wheel and ran a concurrent reproducer (many threads
hammering the client while `atexit` fires) over 100 times with no hangs
or crashes, plus a 30-second-join variant and repeated runs of a
short-lived process confirming clean exits with no added latency.

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-15 12:48:31 -07:00
Jack Ye f12996557f feat(remote): support materialized view APIs (#4180) 
## Summary

Align the experimental materialized-view HTTP transport with the
equivalent Table API shape and add remote materialized-view support
across Rust, Python, and TypeScript. This is an intentional breaking
change to the experimental materialized-view surface.

Materialized-view creation performs an initial refresh by default. The
create endpoint returns `202 Accepted` with `{ "job_id": "..." }`;
blocking SDK creation waits for that job before returning a populated
view. `with_no_data` / `withNoData` explicitly creates only the
definition and empty backing table.

## Route comparison

| Operation | Materialized-view API | Equivalent Table API |
| --- | --- | --- |
| Create | `POST /v1/materialized_view/{id}/create` | `POST
/v1/table/{id}/create` |
| Describe/open | `POST /v1/materialized_view/{id}/describe` | `POST
/v1/table/{id}/describe` |
| List | `GET /v1/namespace/{id}/materialized_view/list` | `GET
/v1/namespace/{id}/table/list` |
| Refresh | `POST /v1/materialized_view/{id}/refresh` | asynchronous
Table mutation pattern |
| Drop | `POST /v1/materialized_view/{id}/drop` | `POST
/v1/table/{id}/drop` |

Create, describe, refresh, and drop identify the target in the singular
item path instead of duplicating it in the request body. Create and drop
require `202 Accepted` with a valid job ID. List is a namespace-scoped
GET with opaque pagination tokens. The Rust list API now returns view
names, matching Table listing and the existing Python and TypeScript
APIs.

## Python API changes

| Operation | Synchronous API | Asynchronous API | Table/job pattern |
| --- | --- | --- | --- |
| Create and wait | `DBConnection.create_materialized_view(...)` |
`await AsyncConnection.create_materialized_view(...)` | Returns a
materialized-view handle after its initial-population job finishes |
| Submit create | `DBConnection.create_materialized_view_async(...) ->
Job[None]` | `await AsyncConnection.create_materialized_view_async(...)
-> AsyncJob[None]` | Matches job-returning Table mutations such as
`create_index_async` |
| Open | `DBConnection.open_materialized_view(...)` | `await
AsyncConnection.open_materialized_view(...)` | Opens the backing Table
plus its definition |
| List | `DBConnection.list_materialized_views()` | `await
AsyncConnection.list_materialized_views()` | Returns names like Table
listing |
| Refresh and wait | `MaterializedView.refresh(...)` | `await
AsyncMaterializedView.refresh(...)` | Returns the typed refresh result
after the job finishes |
| Submit refresh | `MaterializedView.refresh_async(...) ->
Job[RefreshMaterializedViewResult]` | `await
AsyncMaterializedView.refresh_async(...) ->
AsyncJob[RefreshMaterializedViewResult]` | Matches
`Table.refresh_column_async`; remote job handles expose the server job
ID |
| Drop | `DBConnection.drop_materialized_view(...)` | `await
AsyncConnection.drop_materialized_view(...)` | Matches blocking
`drop_table` |
| Submit drop | `DBConnection.drop_materialized_view_async(...) ->
Job[None]` | `await AsyncConnection.drop_materialized_view_async(...) ->
AsyncJob[None]` | Matches `drop_table_async`; remote handles expose the
server cleanup job ID |

The materialized-view handle exposes its backing Table through `.table`,
so normal Table query, search, and index APIs apply. Definition lookup
and refresh are backend-aware rather than depending on local schema
metadata. TypeScript exposes the equivalent blocking/job drop pair as
`dropMaterializedView` and `dropMaterializedViewAsync`.
 2026-09-15 12:13:39 -07:00
Wyatt Alt 2d4622491e fix: keep computed-column freshness across branches and clones (#4188) 
A branch or shallow clone recomputed every computed column on its first
refresh: the input signature carried each data file's raw base id, which
the clone commit rewrites, and the signature sidecar was looked up under
the branch's own tree, where nothing was ever written.

A file is now identified by where its store resolves it, so the source's
own root and a clone's registered reference to that root sign the same
while different bases stay distinct; compaction products are matched the
same way. Sidecars live in one `_computed/` at the table root shared by
main and its branches; a shallow clone reads through the base it was
cloned from and keeps a copy. Pruning collects references across every
branch, and the staleness walk treats the versions a branch does not
hold as unknown. Stamps written before this change no longer match, so
the first refresh after upgrading recomputes once; a deep clone, which
copies no sidecar, still starts over.
 2026-09-15 11:47:39 -07:00
Jack YeandXuanwo 2f88b71c21 feat: add persistent OAuth token cache and session APIs (#4182) 
Stacked on #4173 (diff includes it until that merges; will rebase
after). Addresses the token-cache part of [Colin's
review](https://github.com/lancedb/lancedb/pull/4173#issuecomment-5674048100).

Adds an explicit, opt-in persistent OAuth token cache shared by Rust,
Python, and Node clients, plus `login` / `status` / `logout` session
APIs, so short-lived processes (CLIs, scripts, notebooks) reuse one
session instead of restarting a browser or device flow on every start.

- **Opt-in and minimal**: existing callers stay memory-only and lazy.
Only refresh tokens are persisted (never access tokens, never client
secrets), so there are no local token-expiry decisions to get wrong when
clocks move. Each process start performs one silent refresh grant.
- **Hardened file backend**: private directory (`0700`), per-record
files (`0600`), owner validation, symlink rejection, and atomic `rename`
replacement. Corrupt, truncated, unknown-version, or permission-invalid
records fail with actionable errors naming the file. Native keyring
backends were evaluated (keyring crate routes Linux through D-Bus/zbus:
heavy deps, headless/CI flakiness) and are deferred; the file store is
the explicit opt-in, not a downgrade from a keyring.
- **Cache key**: SHA-256 of the canonical identity (issuer, client ID,
sorted/de-duplicated scopes, flow, public/confidential), so no secret
appears in a filename and distinct identities never collide. Versioned
record schema (`version: 1`). One record per identity: last login wins,
documented.
- **Cross-process rotation locking**: per-key `fs4` file lock (`flock` /
`LockFileEx`) around the refresh critical section — acquire, reread the
durable record, refresh exactly once, atomically store the rotated
refresh token, release. The OS releases locks on process death, so
crashes cannot strand stale locks. Only confirmed
`invalid_grant`/`invalid_token` deletes a record and reauthenticates;
transport, 5xx, 429, and parse failures retain it.
- **Session APIs**: `OAuthSession::login/status/logout` in Rust,
`lancedb.remote.OAuthSession` (async) in Python, `OAuthSession` class in
Node. `status` returns non-secret metadata only. `logout` removes only
the local credential — provider revocation (RFC 7009) is a deliberate
follow-up, and local logout never terminates browser SSO. Azure managed
identity is rejected for persistence (machine identity stays in memory);
client credentials have nothing refreshable to persist and stay
memory-only.
- No CLI binary exists in this repo, so this ships library APIs plus doc
examples in all three languages.

Tests: Rust unit + mock-IdP integration (cache-key
canonicalization/separation, record
versioning/corruption/truncation/symlink/owner/perms, lock serialization
+ release, two concurrent providers proving no `invalid_grant` and
correct rotation, transient-failure retention, `invalid_grant` delete +
reauthenticate, login/status/logout lifecycle, client-credentials no-op,
IMDS rejection, secret redaction); Python lifecycle + a true
two-subprocess cross-process reuse test (second process refreshes once,
never hits the device endpoint); Node lifecycle + device-flow login
test. Local builds were skipped in development; CI validates all
bindings.

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-09-16 02:01:10 +08:00
Jack Ye 575286922b fix: redact authentication headers from debug logs (#4179) 
Prevent bearer tokens, API keys, cookies, and proxy credentials from
appearing in request debug output through a reusable
`redact_sensitive_headers` utility.

The utility is applied after default/configured header construction and
dynamic header merging, so it also covers retry requests. Invalid
dynamic-header values are no longer echoed, and regression coverage
verifies that credentials are redacted while ordinary diagnostic headers
remain visible.

Extracted from the OAuth discussion in #4173, following [Colin's
review](https://github.com/lancedb/lancedb/pull/4173#issuecomment-5674048100).
 2026-09-15 08:31:28 -07:00
dependabot[bot]andXuanwo 09e5418943 build(deps): bump the rust-minor-patch group across 1 directory with 5 updates (#4178) 
Bumps the rust-minor-patch group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [uuid](https://github.com/uuid-rs/uuid) | `1.26.0` | `1.26.1` |
| [serde_with](https://github.com/jonasbb/serde_with) | `3.22.0` |
`3.23.0` |
| [aws-smithy-types](https://github.com/smithy-lang/smithy-rs) | `1.4.8`
| `1.6.3` |
| [napi-derive](https://github.com/napi-rs/napi-rs) | `3.6.3` | `3.6.5`
|
| [napi-build](https://github.com/napi-rs/napi-rs) | `2.4.1` | `2.4.2` |


Updates `uuid` from 1.26.0 to 1.26.1
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/uuid-rs/uuid/releases">uuid's
releases</a>.</em></p>
<blockquote>
<h2>v1.26.1</h2>
<h2>What's Changed</h2>
<ul>
<li>Seat the v7 counter below the version nibble by <a
href="https://github.com/lenamonj"><code>@​lenamonj</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/907">uuid-rs/uuid#907</a></li>
<li>Don't panic in overflowing Timestamp to SystemTime conversion by <a
href="https://github.com/KodrAus"><code>@​KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/909">uuid-rs/uuid#909</a></li>
<li>Prepare for 1.26.1 release by <a
href="https://github.com/KodrAus"><code>@​KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/910">uuid-rs/uuid#910</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/lenamonj"><code>@​lenamonj</code></a>
made their first contribution in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/907">uuid-rs/uuid#907</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/uuid-rs/uuid/compare/v1.26.0...v1.26.1">https://github.com/uuid-rs/uuid/compare/v1.26.0...v1.26.1</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/uuid-rs/uuid/commit/9f927126c89892ddfed6cd2f92df16852f3f9aa6"><code>9f92712</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/910">#910</a> from
uuid-rs/cargo/v1.26.1</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/d4df8f0cd9f461b4ef493254420052ffa5ce6277"><code>d4df8f0</code></a>
prepare for 1.26.1 release</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/5613f2357c1fc06afc5fffd98e96da2ccf25a608"><code>5613f23</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/909">#909</a> from
uuid-rs/fix/ts-conversion-overflow</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/fda00eba938383d242bad33143c8af73227f2a2c"><code>fda00eb</code></a>
don't panic in overflowing Timestamp to SystemTime conversion</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/c82e88ca184e4ab83ce6b4ac0be33d32a0b9c3c4"><code>c82e88c</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/907">#907</a> from
lenamonj/v7-counter-placement</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/ac065a6c17389a67dc6e98ef5f9a2e1d7ad4a670"><code>ac065a6</code></a>
Align the counter diagram</li>
<li><a
href="https://github.com/uuid-rs/uuid/commit/34ec10208d813672928c299146dfe4c18cedcec7"><code>34ec102</code></a>
Seat the v7 counter below the version nibble</li>
<li>See full diff in <a
href="https://github.com/uuid-rs/uuid/compare/v1.26.0...v1.26.1">compare
view</a></li>
</ul>
</details>
<br />

Updates `serde_with` from 3.22.0 to 3.23.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/jonasbb/serde_with/releases">serde_with's
releases</a>.</em></p>
<blockquote>
<h2>serde_with v3.23.0</h2>
<h3>Changed</h3>
<ul>
<li>Update <code>syn</code> and <code>darling</code> dependencies to use
<code>syn</code> v3 (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/992">#992</a>)</li>
<li>Update dev-dependencies to newer versions (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/993">#993</a>)</li>
<li>Update <code>base64</code> to a newer version. This should not have
any API change, but some error messages might change. (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/993">#993</a>)</li>
<li><code>serde_as</code> can now parse <code>cfg_attr(true, ...)</code>
and <code>cfg_attr(false, ...)</code> (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/995">#995</a>)
<code>true</code>/<code>false</code> are new literals as of Rust 1.88
but need to be parsed explicitly with the <code>syn</code> types.
This is used when emitting <code>schemars</code> annotations.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/jonasbb/serde_with/commit/ea5dfdc6fd1b4732188519e871e2fd2a8fe49f88"><code>ea5dfdc</code></a>
Bump version to v3.23.0 (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1005">#1005</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/e52e85b1db55e4dc6305b6b19c2710d6a8b2d433"><code>e52e85b</code></a>
Bump version to v3.23.0</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/39955b6954796d963d02d2f28e0a22087c48fcd4"><code>39955b6</code></a>
Bump rmp dev-dependency (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1004">#1004</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/b51e5fb877c642c47da36c296c222e1d172d75da"><code>b51e5fb</code></a>
Bump rmp dev-dependency</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/ef598c6683d591851e09cac3a4bd767c93bacb9e"><code>ef598c6</code></a>
Use setup-rust-toolchain v2 instead of v1 (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1003">#1003</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/9ea2658f45e763369f18078d80ea3b109d491de8"><code>9ea2658</code></a>
Fix cargo lint about workspace lints being inherited in the test
crate</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/e5ad81af7721bb770772e8e28de32437f3ff217f"><code>e5ad81a</code></a>
Use setup-rust-toolchain v2 instead of v1</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/81001414f153d18fc1aca6c2f52990c557f90471"><code>8100141</code></a>
Bump the github-actions group across 1 directory with 2 updates (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1002">#1002</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/940f4a62a418784e23026953db8a8fed577bff6d"><code>940f4a6</code></a>
Bump jsonschema from 0.49.8 to 0.52.0 (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1001">#1001</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/6f42b94d7bb71b004d0e591a26fbf8d900878c91"><code>6f42b94</code></a>
Bump the github-actions group across 1 directory with 2 updates</li>
<li>Additional commits viewable in <a
href="https://github.com/jonasbb/serde_with/compare/v3.22.0...v3.23.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `aws-smithy-types` from 1.4.8 to 1.6.3
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/smithy-lang/smithy-rs/commits">compare
view</a></li>
</ul>
</details>
<br />

Updates `napi-derive` from 3.6.3 to 3.6.5
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/napi-rs/napi-rs/releases">napi-derive's
releases</a>.</em></p>
<blockquote>
<h2>napi-derive-v3.6.5</h2>
<h3>Other</h3>
<ul>
<li>update Cargo.toml dependencies</li>
</ul>
<h2>napi-derive-v3.6.4</h2>
<h3>Fixed</h3>
<ul>
<li><em>(deps)</em> update rust crate convert_case to 0.12 (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3469">#3469</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/1492b220d5ad01807b2dcbd250e8383f9d738311"><code>1492b22</code></a>
chore: release (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3502">#3502</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/828983aab9b93f516275625f89905fffca459780"><code>828983a</code></a>
fix(napi): return errors from the serde deserializer for unexpected JS
value ...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/606b14238b8d82578705161c0b0d6b6f4b7c2556"><code>606b142</code></a>
fix(napi): validate wrapped payload provenance in
Object::unwrap/remove_wrapp...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/a75d89f85c0a9a0ffd02b8bb11a2c0897b72a2ae"><code>a75d89f</code></a>
fix(napi): point from_external slices at the engine-owned copy after
finalize...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/e414c8c21924a4c47bf1c8c5d6ad244dac479578"><code>e414c8c</code></a>
fix(deps): update dependency obug to v3 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3499">#3499</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/a5fedde2597dc3ac17c7d422f158adce5fec59bf"><code>a5fedde</code></a>
chore(deps): update release-plz/action action to v0.5.136 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3505">#3505</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/31c27a1676a7c4b317f4e144e0a9cb94e8354143"><code>31c27a1</code></a>
chore: release (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3470">#3470</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/7e3f293e2d6a3032eabfe51ff38bcaa82d342a2f"><code>7e3f293</code></a>
chore(release): publish</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/f772ee0aabf4dbb27250a8df47a73463e8f78cf4"><code>f772ee0</code></a>
fix(cli): align generated file formats (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3501">#3501</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/1cf5ec573637d96dddb77b1a0ae6aaf316739bd0"><code>1cf5ec5</code></a>
fix(cli): use accessible WASI preopen root on Android (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3485">#3485</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/napi-rs/napi-rs/compare/napi-derive-v3.6.3...napi-derive-v3.6.5">compare
view</a></li>
</ul>
</details>
<br />

Updates `napi-build` from 2.4.1 to 2.4.2
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/napi-rs/napi-rs/releases">napi-build's
releases</a>.</em></p>
<blockquote>
<h2>napi-build-v2.4.2</h2>
<h3>Fixed</h3>
<ul>
<li><em>(cli,build)</em> make wasm32-wasip1-threads link with wasi-sdk
34 and Rust nightly (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3492">#3492</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/31c27a1676a7c4b317f4e144e0a9cb94e8354143"><code>31c27a1</code></a>
chore: release (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3470">#3470</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/7e3f293e2d6a3032eabfe51ff38bcaa82d342a2f"><code>7e3f293</code></a>
chore(release): publish</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/f772ee0aabf4dbb27250a8df47a73463e8f78cf4"><code>f772ee0</code></a>
fix(cli): align generated file formats (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3501">#3501</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/1cf5ec573637d96dddb77b1a0ae6aaf316739bd0"><code>1cf5ec5</code></a>
fix(cli): use accessible WASI preopen root on Android (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3485">#3485</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/cf9245d0b367a8fc2b3c1209269832b9f076de32"><code>cf9245d</code></a>
chore(deps): update vitest monorepo to v5 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3481">#3481</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/78fa3ad79d38b52e7c49f9a86ba2e57f46805d1b"><code>78fa3ad</code></a>
fix(macro): recognize fully-qualified napi::Env as the special Env
parameter ...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/37283c75a1f1f20d166a8f9f641ada43d04d24a5"><code>37283c7</code></a>
chore(deps): lock file maintenance (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3474">#3474</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/1932d2f42198cb9f79704aba66302a0956590c48"><code>1932d2f</code></a>
chore(deps): update release-plz/action action to v0.5.135 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3500">#3500</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/0238e8b1b8fcafa2261d4d1b96e7568dd0fb1fc3"><code>0238e8b</code></a>
fix(deps): update dependency js-yaml to v5 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3344">#3344</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/9ed9d3509e1c2cab2cd89dcff04963adaad4cb7d"><code>9ed9d35</code></a>
chore(deps): update dependency electron to v44 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3468">#3468</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/napi-rs/napi-rs/compare/napi-build-v2.4.1...napi-build-v2.4.2">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-09-15 22:00:31 +08:00
LanceDB RobotYang CenClaude Fable 5.1
 		21ecafe9ae chore: update lance dependency to v13.0.0-beta.1 (#4183) 
Update the Rust workspace Lance dependencies and Java lance-core from
v12.0.0-beta.18 to
[v13.0.0-beta.1](https://github.com/lance-format/lance/releases/tag/v13.0.0-beta.1),
and refresh Cargo.lock; no compatibility fixes were required.

Validation passed: `cargo clippy --quiet --workspace --tests
--all-features -- -D warnings`, `cargo fmt --all --quiet`, and `git diff
--check`.

Also fixes the flaky Node test `when optimizing a dataset › cleanups old
versions` that failed the NPM Publish Linux test jobs on this PR. The
test captured `new Date()` (millisecond precision) in the same
millisecond as the last commit, while Lance compares version timestamps
at nanosecond precision, so that version was not pruned. The test now
waits for the clock to tick to the next millisecond before taking the
cutoff. This is a pre-existing flake on `main` since #4160, unrelated to
the Lance upgrade.

---------

Co-authored-by: Yang Cen <bubble-cal@outlook.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-15 19:13:44 +08:00
Jack YeandXuanwo 56bbd19ff4 test(python): cover namespace operations without pylance (#4174) 
The "Test without pylance or pandas" CI job only ran `test_table.py`, so
the
regression fixed in #3606 had no guard: sync namespace operations used
to route
through the Python `lance_namespace` client, whose `dir` implementation
ships in
the optional `pylance` extra, so
`lancedb.connect(path).list_namespaces()` failed
with `No module named 'lance'` while the async API worked.

Adds `python/python/tests/test_namespace_no_pylance.py` and runs it in
that
job. Its `without_pylance` fixture blocks `lance` imports, so the guard
also
fires in environments that do have `pylance` installed. Coverage: the
original
reproducer, nested namespace lifecycle, namespaced and root table
lifecycle, the
async path, and the one API that legitimately still needs `pylance`
(`namespace_client()`).

Verified the guard actually catches the regression: against
`lancedb==0.33.0`,
three of these tests fail with the original error; against a fixed build
all
pass.

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-09-15 17:10:49 +08:00
lancedb-gatefixer[bot]andXuanwo ffe94a65a1 fix(node): preserve optimize cleanup timestamp (#4160) 
<!-- lance-gatekeeper-fix:v1 agent=278cb095b2e5d69051442bc254d803a6
generation=1 -->

## Summary

- pass the TypeScript `cleanupOlderThan` date to the native binding as
an unchanged epoch timestamp
- prune with Lance's absolute `before_timestamp` policy so dispatch and
compaction time cannot move the cutoff
- retain versions created after the supplied cutoff and document that
behavior
- add boundary and end-to-end regression coverage

## Root cause

The TypeScript layer converted the absolute date into an elapsed
duration before calling native optimize. Lance converted that duration
back into a timestamp only after compaction, which silently advanced the
requested cutoff and made the cleanup count depend on a millisecond
timing boundary.

## Validation

- `cargo fmt --all`
- `cargo clippy --quiet --features remote --tests --examples -p lancedb
-p lancedb-nodejs`
- `pnpm build`
- `pnpm lint`
- `pnpm run docs`
- `pnpm test __test__/table.test.ts --runInBand` (309 passed)

Fixes #4159

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-09-15 14:10:48 +08:00
Jack Ye 37771fd4fc fix(deps): update rustls and cap aws-smithy-types to unbreak CI (#4177) 
Two upstream dependency releases broke CI on `main`. Both fixes are
dependency constraints, so they ride together.

## `deny` — RUSTSEC-2026-0285

rustls 0.23.40 accepts TLS 1.3 handshake messages sent at the wrong
encryption level
([advisory](https://rustsec.org/advisories/RUSTSEC-2026-0285)), patched
in 0.23.45.

rustls 0.23.45 requires `aws-lc-rs >= 1.18`, which the nodejs crate
pinned to `=1.16.3`, so this also bumps that pin and its `aws-lc-sys`
companion to `=1.18.1` / `=0.45.0`. The pin comment already calls for
periodic updates on security patches.

The workspace's other rustls (0.21.12) is below the advisory's affected
range (`unaffected = ["< 0.23.13"]`).

## `build-no-lock` — aws-smithy-types 1.7.0

`aws-smithy-types` 1.7.0 and `aws-smithy-json` 0.64.0 both released
2026-09-14. 1.7.0 made `Document` `non_exhaustive`, which
`aws-smithy-json` 0.63 does not compile against:

```
error[E0004]: non-exhaustive patterns: `&_` not covered
  --> aws-smithy-json-0.63.0/src/serialize.rs:36:15
note: `aws_smithy_types::Document` defined here
  --> aws-smithy-types-1.7.0/src/document/mod.rs:91:1
```

Every `aws-sdk-*` crate moved to `aws-smithy-json ^0.64`, but
`aws-config` 1.12.0 still requires `^0.63`, so a lockfile-free resolve
pairs json 0.63.0 with types 1.7.0 and fails. This caps
`aws-smithy-types` below 1.7 as a constraint-only dev-dependency,
matching the existing `aws-smithy-runtime` entry. Revert once
`aws-config` moves to `aws-smithy-json` 0.64.

Note this break is not specific to this PR — `build-no-lock` fails the
same way on unrelated branches (e.g. `jon/secrets-client-api` run
34903587364), which passed it hours earlier.

## Verification

Resolution only, no local build:

- Locked resolve unchanged: `aws-smithy-types` stays 1.4.8; the only
`Cargo.lock` delta from the cap is the new dev-dep edge.
- Fresh resolve (`rm Cargo.lock`): `aws-smithy-json` 0.63.0 with
`aws-smithy-types` 1.6.3, `aws-sdk-*` one release back, `rustls` 0.23.45
retained.
 2026-09-14 20:04:15 -07:00
Lance Release c44b192334 Bump version: 0.39.0-beta.7 → 0.39.0-beta.8 2026-09-14 16:46:53 +00:00
Wyatt Alt 7575c2597a feat: recompute computed column rows whose inputs changed (#4161) 
refresh_column fills nulls, so once a row has a value nothing revisits
it: an update to one of its inputs, or a definition change, leaves the
computed value stale for good.

This stamps the column's field metadata with the definition it was
computed under and a per-fragment signature of the input storage it was
read from (input data files and overlays; not the deletion file, since a
delete changes no surviving value). A refresh recomputes every live row
of a fragment whose stamp disagrees with the manifest, then records what
it computed from in a second commit after the fill. A compacted fragment
inherits freshness through the Rewrite lineage when every fragment it
was built from was signed, or was appended since the stamp, never had an
input moved, and left its rows of the product unfilled (a raw append may
supply a value; the product's data is the evidence, and the null fill
covers those rows); otherwise it recomputes. A column declared before
the stamps existed keeps the null-fill contract on its first refresh,
which enrolls it as it stood.

The map is one entry per fragment per column, so it is kept out of the
manifest: each stamp writes an immutable sidecar under `_computed/`,
named by its content digest, and the field metadata holds the digest.
Pruning old versions also drops the sidecars no remaining version
references, keeping any younger than seven days as lance keeps
unverified files, since a sidecar is put before the commit that
references it. The stamp commit is metadata-only, so a materialized
view's drift check treats it like the fill. The core lives in
`table::freshness` so a remote refresh can share the contract.
 2026-09-14 09:41:37 -07:00
Wyatt Alt 0665575a76 feat: recompute computed column rows whose inputs changed (#4161) 
refresh_column fills nulls, so once a row has a value nothing revisits
it: an update to one of its inputs, or a definition change, leaves the
computed value stale for good.

This stamps the column's field metadata with the definition it was
computed under and a per-fragment signature of the input storage it was
read from (input data files and overlays; not the deletion file, since a
delete changes no surviving value). A refresh recomputes every live row
of a fragment whose stamp disagrees with the manifest, then records what
it computed from in a second commit after the fill. A compacted fragment
inherits freshness through the Rewrite lineage when every fragment it
was built from was signed, or was appended since the stamp, never had an
input moved, and left its rows of the product unfilled (a raw append may
supply a value; the product's data is the evidence, and the null fill
covers those rows); otherwise it recomputes. A column declared before
the stamps existed keeps the null-fill contract on its first refresh,
which enrolls it as it stood.

The map is one entry per fragment per column, so it is kept out of the
manifest: each stamp writes an immutable sidecar under `_computed/`,
named by its content digest, and the field metadata holds the digest.
Pruning old versions also drops the sidecars no remaining version
references, keeping any younger than seven days as lance keeps
unverified files, since a sidecar is put before the commit that
references it. The stamp commit is metadata-only, so a materialized
view's drift check treats it like the fill. The core lives in
`table::freshness` so a remote refresh can share the contract.
 2026-09-14 09:39:55 -07:00
陈志谦andXuanwo 0113cee489 docs(embeddings): correct the documented max_retries default (#4145) 
`lancedb/embeddings/utils.py` documents `max_retries` with "(default is
10)" — the signature default is `7`. Docs-only; conventional title per
the contribution guide.

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-09-14 22:40:50 +08:00
lancedb-gatefixer[bot]andXuanwo 6bb64c3edb fix: reject repeated job pagination tokens (#4139) 
Fixes #4138

`RemoteDatabase::list_jobs` followed every returned pagination token
without remembering previously seen values. A server-side token cycle
therefore caused repeated requests and duplicate accumulation until the
100-page safeguard returned partial results as a success.

This change tracks non-empty job-list page tokens and returns an
HTTP-context error as soon as a token repeats, matching the existing
`list_functions` behavior. A mock-handler regression test verifies a
repeated `loop` token is rejected after two requests.

Validation:
- `cargo test --quiet --features remote -p lancedb test_list_jobs`
- `cargo fmt --all`
- `cargo check --quiet --features remote --tests --examples`

<!-- lance-gatekeeper-fix:v1 agent=ddd56389737a405d32cf4f43c7696032
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-09-14 20:50:54 +08:00
Xuanwo 255da8a854 fix(python): resolve native job types in API docs (#4170) 
Native job metadata types report `builtins` as their module, so Griffe
cannot resolve the public `lancedb.job` re-exports and the Python API
reference build fails. Set the PyO3 module metadata for `JobInfo`,
`JobDescription`, and `JobFailureInfo`, and cover import resolution in
the existing package metadata tests.

Reproduced the failure and validated the fix with the docs CI toolchain
(`griffe==0.49.0`, `mkdocstrings==0.25.2`, and
`mkdocstrings-python==1.10.9`). After rebuilding the native extension,
the full `PYTHONPATH=. mkdocs build` succeeds and all three classes and
their public members appear in the generated reference.
 2026-09-14 15:09:29 +08:00
Lance Release ec410e015a Bump version: 0.39.0-beta.6 → 0.39.0-beta.7 2026-09-14 04:45:33 +00:00
Jack Ye 9fe10c7362 fix(remote): align Function CRUD routes (#4166) 
Align the experimental Function HTTP transport with the equivalent Table
CRUD API shape. This is an intentional breaking change to the
experimental Function routes; public Rust and Python APIs remain
unchanged.

## Route comparison

| Operation | Function before | Function after | Equivalent Table API |
| --- | --- | --- | --- |
| Create | `POST /v1/functions/create` | `POST /v1/function/{id}/create`
| `POST /v1/table/{id}/create` |
| Describe | `POST /v1/functions/describe` | `POST
/v1/function/{id}/describe` | `POST /v1/table/{id}/describe` |
| List | `POST /v1/functions/list` | `GET
/v1/namespace/{id}/function/list` | `GET /v1/namespace/{id}/table/list`
|
| Drop | `POST /v1/functions/drop` | `POST /v1/function/{id}/drop` |
`POST /v1/table/{id}/drop` |

## Contract details

- Create, describe, and drop use a singular resource path. Their `{id}`
path parameter is the URL-encoded Function name, and the duplicate
Function identifier is removed from each request body.
- Create continues to accept `202 Accepted`.
- List changes from a POST with a JSON body to a namespace-scoped GET.
Its `{id}` path parameter is the namespace identifier rather than a
Function name.
- Functions do not support nested namespaces yet, so the client lists
against the root namespace identifier (`$` with the default delimiter).
A non-root namespace is rejected.
- The optional list filter is named `name`. `limit`, `page_token`, and
`include_definition` remain available as query parameters.
- The paginated list response shape is unchanged.
 2026-09-13 21:44:13 -07:00
LanceDB Robot b8f0048b5a chore: update lance dependency to v12.0.0-beta.18 (#4164) 
Update the Rust workspace Lance dependencies, Cargo lockfile, and Java
lance-core to
[v12.0.0-beta.18](https://github.com/lance-format/lance/releases/tag/v12.0.0-beta.18).
Fix redundant visibility declarations in the Node.js Rust blob helpers
required by the workspace Clippy check.
Validated with workspace Clippy (all features and tests, warnings
denied), cargo fmt, pnpm build, and 13 targeted Node.js blob tests.
 2026-09-12 22:25:12 -07:00
Drew 6702e3fec1 feat(node): add blob v2 fetch and field helpers (#4155) 
this PR blob v2 field helpers and reads to the Node SDK.

`blob()` marks a field as blob v2 and lets you set the storage
thresholds. Inputs can be bytes, a URI, or a data/uri struct.

Queries return descriptors. `fetchBlobs()` reads the bytes by row ID,
and `fetchBlobFiles()` gives you lazy handles for full or range reads.
`blobColumns()` lists the blob fields, including nested ones.

Fetch uses the table’s current checkout. It preserves order, duplicates,
and nulls. Holding row IDs across compaction still requires stable row
IDs.

```javascript
const db = await connect("./data");
const video = await readFile("clip.mp4");
const table = await db.createTable(
  "videos",
  [{ id: 1n, video }],
  {
    schema: new Schema([
      new Field("id", new Int64()),
      blob("video"),
    ]),
  },
);
const rows = await table.query().select(["id"]).withRowId().toArray();
const rowIds = rows.map((row) => row._rowid as bigint);
const bytes = await table.fetchBlobs("video", rowIds);
const [handle] = await table.fetchBlobFiles("video", rowIds);
const header = await handle!.readRange(0n, 65536n);

```

### Testing

- cover input validation, thresholds, nested fields, fetch ordering,
nulls, and range reads.
 2026-09-11 14:46:27 -07:00
LanceDB Robot e0bd4b5fa1 chore: update lance dependency to v12.0.0-beta.17 (#4162) 
Update the Rust workspace Lance dependencies and Java lance-core to
[v12.0.0-beta.17](https://github.com/lance-format/lance/releases/tag/v12.0.0-beta.17).
Align object_store to 0.14.1 for compatibility with Lance and refresh
the Cargo lockfile, including the required reqsign updates. Validation
passed: `cargo clippy --quiet --workspace --tests --all-features -- -D
warnings` and `cargo fmt --all --quiet`.
 2026-09-11 09:59:19 -07:00
dependabot[bot] 13f9dd630b build(deps): bump prost from 0.14.3 to 0.14.4 in the rust-minor-patch group (#4135) 
Bumps the rust-minor-patch group with 1 update:
[prost](https://github.com/tokio-rs/prost).

Updates `prost` from 0.14.3 to 0.14.4
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/prost/blob/master/CHANGELOG.md">prost's
changelog</a>.</em></p>
<blockquote>
<h1>Prost version 0.14.4</h1>
<p><em>PROST!</em> is a <a href="https://protobuf.dev/">Protocol
Buffers</a> implementation for the <a
href="https://www.rust-lang.org/">Rust Language</a>. <code>prost</code>
generates simple, idiomatic Rust code from <code>proto2</code> and
<code>proto3</code> files.</p>
<h3>🚀 Features</h3>
<ul>
<li><em>(prost-derive)</em> Make is_valid a constant function (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1401">#1401</a>)</li>
<li>Increase MSRV to 1.85 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1428">#1428</a>)</li>
</ul>
<h3>🐛 Bug Fixes</h3>
<ul>
<li>Use Display instead of Debug for generated enumeration attributes
(<a
href="https://redirect.github.com/tokio-rs/prost/issues/1419">#1419</a>)</li>
<li><em>(prost-derive)</em> Return error for invalid enumeration default
identifiers (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1426">#1426</a>)</li>
<li><em>(build)</em> Grab binary path from cargo (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1429">#1429</a>)</li>
<li><em>(build)</em> Fix C++ build on GCC 15 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1395">#1395</a>)</li>
</ul>
<h3>📚 Documentation</h3>
<ul>
<li>Add example for <code>decode_length_delimiter</code> (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1311">#1311</a>)</li>
<li>Update protobuf-src example to avoid unsafe set_var</li>
</ul>
<h3>🧪 Testing</h3>
<ul>
<li>Test derive Eq behavior (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1422">#1422</a>)</li>
<li><em>(groups)</em> Actually construct <code>NestedGroup</code> (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1363">#1363</a>)</li>
</ul>
<h3>💼 Dependencies</h3>
<ul>
<li><em>(deps)</em> Update criterion requirement from 0.7 to 0.8 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1374">#1374</a>)</li>
<li><em>(deps)</em> Remove <code>getrandom@0.4.1</code> from
build-dependencies (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1400">#1400</a>)</li>
<li><em>(deps)</em> Update rand requirement from 0.9 to 0.10 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1397">#1397</a>)</li>
<li><em>(deps)</em> Bump actions/upload-artifact from 6 to 7 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1409">#1409</a>)</li>
<li><em>(deps)</em> Update <code>cargo clippy</code> to 1.89 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1433">#1433</a>)</li>
<li><em>(deps)</em> Update <code>cargo clippy</code> to 1.91 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1435">#1435</a>)</li>
<li><em>(deps)</em> Update and improve nix devshell (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1393">#1393</a>)</li>
</ul>
<h3>🎨 Styling</h3>
<ul>
<li>Prevent needless borrow (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1404">#1404</a>)</li>
<li>Use <code>std::hint::black_box()</code> (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1403">#1403</a>)</li>
<li>Use variables directly in <code>format!()</code> (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1432">#1432</a>)</li>
<li>Remove explicit <code>.into_iter()</code> (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1434">#1434</a>)</li>
<li>Run clippy on benches (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1405">#1405</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/tokio-rs/prost/commit/13646cde7eab75c81b3047767aa0a86e7dbecf12"><code>13646cd</code></a>
chore: Release version 0.14.4 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1437">#1437</a>)</li>
<li><a
href="https://github.com/tokio-rs/prost/commit/dad79d5c8e3549d93ebe6f6c723bb42928d805d8"><code>dad79d5</code></a>
fix(prost-derive): return error for invalid enumeration default
identifiers (...</li>
<li><a
href="https://github.com/tokio-rs/prost/commit/b0b6c93e3aac89df28690a4967a8bbe93ec95391"><code>b0b6c93</code></a>
ci: Update <code>cargo clippy</code> to 1.91 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1435">#1435</a>)</li>
<li><a
href="https://github.com/tokio-rs/prost/commit/32cfffbc494f2faf461cab85e04a42412484c0e4"><code>32cfffb</code></a>
style: remove explicit <code>.into_iter()</code> (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1434">#1434</a>)</li>
<li><a
href="https://github.com/tokio-rs/prost/commit/2710efdb9978d9c75fb19b0b092a369a2d385b55"><code>2710efd</code></a>
ci: Update <code>cargo clippy</code> to 1.89 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1433">#1433</a>)</li>
<li><a
href="https://github.com/tokio-rs/prost/commit/18ea4e42bbc307d33d65e05ad47b3c45623c0500"><code>18ea4e4</code></a>
style: use variables directly in <code>format!()</code> (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1432">#1432</a>)</li>
<li><a
href="https://github.com/tokio-rs/prost/commit/2821bd1d8c20137c83ead4db39f8e1da00b4e854"><code>2821bd1</code></a>
build(deps): bump actions/upload-artifact from 6 to 7 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1409">#1409</a>)</li>
<li><a
href="https://github.com/tokio-rs/prost/commit/3ce3b39f9206b5e3bbe34c6e1aa69fe3c53f0924"><code>3ce3b39</code></a>
test(groups): Actually construct <code>NestedGroup</code> (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1363">#1363</a>)</li>
<li><a
href="https://github.com/tokio-rs/prost/commit/8776405574b3ba0a0fe96ada8799ac8bc61ceb3e"><code>8776405</code></a>
docs: Update changelog for version 0.14.3 (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1431">#1431</a>)</li>
<li><a
href="https://github.com/tokio-rs/prost/commit/33d3ef18c008da13e862d7e7674d751ab2776360"><code>33d3ef1</code></a>
build: Grab binary path from cargo (<a
href="https://redirect.github.com/tokio-rs/prost/issues/1429">#1429</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/tokio-rs/prost/compare/v0.14.3...v0.14.4">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=prost&package-manager=cargo&previous-version=0.14.3&new-version=0.14.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-10 10:15:08 -07:00
LanceDB Robot bc4497b21a chore: update lance dependency to v12.0.0-beta.16 (#4156) 
Updates the Rust workspace Lance dependencies, Cargo lockfile, and Java
lance-core from v12.0.0-beta.15 to
[v12.0.0-beta.16](https://github.com/lance-format/lance/releases/tag/v12.0.0-beta.16).
No compatibility fixes were required; `cargo clippy --quiet --workspace
--tests --all-features -- -D warnings`, `cargo fmt --all --quiet`, and
`git diff --check` passed.
 2026-09-10 00:32:31 -07:00
Will JonesandClaude Sonnet 5 1da5876870 ci: add spell checking (#4148) 
Adds [typos](https://github.com/crate-ci/typos) as a CI check and
pre-commit hook, the same way Lance does it, so misspellings like the
ones fixed in #4146 get caught automatically going forward.

This also fixes the misspellings `typos` found across the repo (Rust,
Python, TypeScript source, comments, and generated docs), and adds a
small `.typos.toml` with `extend-words` entries for terms that are
correct but look like typos: `AKS` (Azure Kubernetes Service), `RabitQ`
(a real quantization algorithm name), `mmaped` (the actual name of a
`candle-core` API we call), and `Writeable` (from Python's
`_typeshed.WriteableBuffer`). Third-party license files are excluded.

Fixes #4147

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-09 15:33:04 +08:00
Madan Kumar 577fb48376 fix(python): apply offset when combining async hybrid results (#4028) 
Fixes #4027

## Summary

`AsyncHybridQuery`
(`table.query().nearest_to(...).nearest_to_text(...)`) paginates
incorrectly when `.offset()` is used: the second page repeats rows from
the first page and silently drops others.

`offset()` on a hybrid query pushes the offset down into *both*
sub-queries (`HybridQuery::offset` in `python/src/query.rs` forwards to
`inner_vec` and `inner_fts`), so each sub-query independently skips its
own first `offset` rows before the results are fused.
`AsyncHybridQuery.to_batches` then called `_combine_hybrid_results(...,
limit=self._inner.get_limit())` without an `offset`, so the reranked
table was sliced starting at position 0 and the sub-query limits were
never raised to cover the skipped prefix.

On the 4-row fixture in `test_hybrid_query.py`, with `_rowid` ordering
`[3, 0, 2, 1]`:

| query | before | after |
| --- | --- | --- |
| `.limit(2)` | `[0, 3]` | `[0, 3]` |
| `.offset(2).limit(2)` | `[3, 1]` | `[2, 1]` |

Row `3` was returned on both pages and row `2` was never returned at
all.

This is the async counterpart of #3769 (`Fixes #3765`), which fixed the
same bug in the synchronous `LanceHybridQueryBuilder`. #3765 explicitly
deferred the async path; this PR closes that gap and reuses the `offset`
parameter that #3769 already added to `_combine_hybrid_results`. The
synchronous path is unaffected — it was fixed in #3769.

## Changes

`python/python/lancedb/query.py`, `AsyncHybridQuery.to_batches`:

- Each sub-query now fetches `limit + offset` rows and its own offset is
reset to 0, so the fused result contains the full prefix the window is
sliced out of.
- The combined, reranked table is sliced with `offset=` instead of
always starting at 0.

Both halves are needed: raising the sub-query limits without the final
slice still returns page 1, and slicing without raising the limits still
misses rows.

`nodejs` has no equivalent hybrid combine path, so there is no SDK
parity gap here.

## Test plan

- [x] New regression test `test_async_hybrid_query_offset` in
`python/python/tests/test_hybrid_query.py`, mirroring the sync
`test_hybrid_query_offset`. It asserts the offset window is a suffix of
the un-offset result *and* that page 1 + page 2 together cover every row
exactly once (a row-count-only assertion would pass even with
duplicates).
- [x] `pytest python/tests/test_hybrid_query.py` — 16 passed
- [x] `pytest python/tests/test_rerankers.py` — 9 passed, 11 skipped
- [x] `pytest python/tests/test_query.py` — 86 passed
- [x] `pytest --doctest-modules python/lancedb/query.py` — 13 passed
- [x] `ruff format --check` / `ruff check` — clean
---

## Scope, after review

@lancedb-gatekeeper raised three points. Two were mine and are fixed in
`04d07c2`; the third is deliberately left alone and I'd like a
maintainer's call on it.

**Fixed — effective limit was read from the FTS child only.**
`HybridQuery::get_limit()` (`python/src/query.rs:1159`) returns
`self.inner_fts.inner.current_request().limit`, so an FTS-first hybrid
with no explicit `.limit()` yielded `None`, skipped the widening branch
and passed `limit=None` to the combiner — returning the union of both
candidate lists instead of the documented default of 10. The limit is
now derived from both children with a `DEFAULT_HYBRID_LIMIT = 10`
fallback, so construction order no longer matters.

**Fixed — `explain_plan()` / `analyze_plan()` described a different
query than the one that ran.** Both built their children straight from
`self._inner`, bypassing the limit/offset rewrite in `to_batches`, and
reported `skip=2, fetch=2` while execution used `skip=0, fetch=4`. Child
preparation now lives in one `_create_child_queries()` helper used by
all three.

> **Visible change to `explain_plan()` output:** because the plan is now
built from the real execution children, which carry `with_row_id()`, the
two `ProjectionExec` lines gain a `_rowid` column. The doctest is
updated to match. This is the diagnostic becoming truthful rather than
the assertion being weakened — it is still an exact-match comparison.

**Not fixed here — RRF candidate-pool invariance.** Widening each
sub-query to `limit + offset` does change the candidate pool between
page requests, so the fused ranking can shift and pagination can still
repeat rows. That's a real problem, but it is exactly what the merged
sync path does today:

```python
# LanceHybridQueryBuilder (sync), merged in #3769
sub_query_limit = self._limit + (self._offset or 0)
```

Making the pool invariant means choosing a contract — a fixed candidate
pool, or an explicit cursor — and that ought to apply to sync and async
together rather than letting the two paths diverge. I've asked in the
review thread which way you'd prefer, and I'm happy to do it here or in
a follow-up covering both paths.

So, to be precise about what this PR delivers: it makes `.offset()` take
effect on the async hybrid path and makes the diagnostics honest. It
does not make hybrid pagination stable across pages under reranking —
that needs the contract decision above.
 2026-09-08 15:42:06 -07:00
陈志谦 c7b051aff7 docs: fix spelling typos across python package docstrings (#4146) 
Six files carried spelling typos in user-visible docstrings:

- `table.py` (×3) + `remote/table.py`: "The **targetted** vector to
search for" → "targeted"
- `query.py`: "pa.Array **wouln't** be allowed" → "wouldn't"
- `embeddings/gte.py`: "mlx package **insalled**" → "installed"
- `rerankers/base.py`: "This is **inteded**" → "intended"
- `index.py`: "dimension **divded** by 8" → "divided"

Docstrings only.
 2026-09-08 13:53:38 -07:00
Lance Release 2e205ac9bb Bump version: 0.39.0-beta.5 → 0.39.0-beta.6 2026-09-08 12:14:45 +00:00
LanceDB RobotandJack Ye 3e3878b223 chore: update lance dependency to v12.0.0-beta.15 (#4143) 
Updates the Rust workspace Lance dependencies, Cargo lockfile, and Java
lance-core from v12.0.0-beta.14 to
[v12.0.0-beta.15](https://github.com/lance-format/lance/releases/tag/v12.0.0-beta.15).
No compatibility fixes were required; `cargo clippy --quiet --workspace
--tests --all-features -- -D warnings`, `cargo fmt --all --quiet`, and
`git diff --check` passed.

---------

Co-authored-by: Jack Ye <yezhaoqin@gmail.com>
 2026-09-08 05:12:45 -07:00
Lance Release 19fb665c76 Bump version: 0.39.0-beta.4 → 0.39.0-beta.5 2026-09-08 12:03:46 +00:00
Wyatt AltandClaude Fable 5.1 1f95398c34 feat: function columns on materialized views (#4119) 
A view could not carry a column it does not compute: the definition
planned every output as a SQL expression, and the refresh engine treated
any commit it did not make as drift and rebuilt. Servers fill such
columns on tables with a separate job, as computed columns bound to a
registered function, and want the same column on a view.

This lets a declaration add computed columns, placed at their positions
in the select list and validated by the existing computed-column
contract, with the view created in one commit. Refresh writes those
columns NULL on every path and never reads them, so a rewritten row
comes back unfilled, and a commit that rewrites only computed columns is
recognised as a fill rather than drift, so the next refresh carries on
incrementally. A source column a computed column reads without the view
projecting it is held as an internal projection, so the select list
stays the view's column list. Nothing in the stored definition changes;
an older reader fails closed on the schema check.

Two smaller changes ride along because the feature needs them: an
identity projection keeps its source column's nullability, with the
schema check accepting a nullable physical field for a non-null planned
one so existing views keep refreshing; and `prepare_declaration` takes
`Option` projections, so an empty list declares no projection rather
than `SELECT *`.

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 04:38:22 -07:00
Lance Release 0111a72dc3 Bump version: 0.39.0-beta.3 → 0.39.0-beta.4 2026-09-06 20:46:55 +00:00
LanceDB RobotandJack Ye a487d4033e chore: update lance dependency to v12.0.0-beta.14 (#4141) 
Update the Rust workspace Lance dependencies and Java lance-core from
v12.0.0-beta.11 to
[v12.0.0-beta.14](https://github.com/lance-format/lance/releases/tag/v12.0.0-beta.14),
refreshing Cargo.lock. Resolve two Clippy diagnostics by making an
internal Node.js helper private and using a byte string literal in a
remote-table test fixture.

Validation: `cargo clippy --quiet --workspace --tests --all-features --
-D warnings`, `cargo fmt --all --quiet`, `git diff --check`, and `pnpm
build` in nodejs.

---------

Co-authored-by: Jack Ye <yezhaoqin@gmail.com>
 2026-09-06 13:43:35 -07:00
dependabot[bot] 02ea0dda9f build(deps-dev): bump the nodejs-deps group across 1 directory with 2 updates (#4134) 
Bumps the nodejs-deps group with 2 updates in the /nodejs directory:
[@opentelemetry/sdk-metrics](https://github.com/open-telemetry/opentelemetry-js)
and [ts-jest](https://github.com/kulshekhar/ts-jest).

Updates `@opentelemetry/sdk-metrics` from 2.10.0 to 2.11.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/open-telemetry/opentelemetry-js/releases">@​opentelemetry/sdk-metrics's
releases</a>.</em></p>
<blockquote>
<h2>v2.11.0</h2>
<h2>2.11.0</h2>
<h3>🚀 Features</h3>
<ul>
<li>feat(context-async-hooks): implement <code>attach()</code> on
<code>AsyncLocalStorageContextManager</code> <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6845">#6845</a>
<a href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a>
<ul>
<li>On Node.js 25.9+, delegates to
<code>AsyncLocalStorage.withScope()</code> returning a native
<code>RunScope</code>. On older Node.js, falls back to
<code>enterWith()</code> with a manual disposable wrapper.</li>
</ul>
</li>
<li>feat(sdk-trace): allow configuring the force flush timeout per call
<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6929">#6929</a>
<a
href="https://github.com/LarryHu0217"><code>@​LarryHu0217</code></a></li>
</ul>
<h3>🐛 Bug Fixes</h3>
<ul>
<li>fix(sdk-metrics): ignore <code>Infinity</code> in exponential
histograms <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7015">#7015</a>
<a href="https://github.com/mwear"><code>@​mwear</code></a></li>
</ul>
<h3>🏠 Internal</h3>
<ul>
<li>perf(sdk-metrics): reuse a single DataView for exponential histogram
bit reads <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6998">#6998</a>
<a href="https://github.com/mwear"><code>@​mwear</code></a></li>
<li>chore(ci): run documentation tests on a weekly schedule <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6920">#6920</a>
<a
href="https://github.com/LarryHu0217"><code>@​LarryHu0217</code></a></li>
<li>feat(ci): support pre-releases and major version bumps in the
release workflow <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6768">#6768</a>
<a
href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a></li>
<li>chore(resources): Ensure that multiple uses of
serviceInstanceIdDetector.detect() return the <em>same</em> value for
<code>service.instance.id</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md">@​opentelemetry/sdk-metrics's
changelog</a>.</em></p>
<blockquote>
<h2>2.11.0</h2>
<h3>🚀 Features</h3>
<ul>
<li>feat(context-async-hooks): implement <code>attach()</code> on
<code>AsyncLocalStorageContextManager</code> <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6845">#6845</a>
<a href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a>
<ul>
<li>On Node.js 25.9+, delegates to
<code>AsyncLocalStorage.withScope()</code> returning a native
<code>RunScope</code>. On older Node.js, falls back to
<code>enterWith()</code> with a manual disposable wrapper.</li>
</ul>
</li>
<li>feat(sdk-trace): allow configuring the force flush timeout per call
<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6929">#6929</a>
<a
href="https://github.com/LarryHu0217"><code>@​LarryHu0217</code></a></li>
</ul>
<h3>🐛 Bug Fixes</h3>
<ul>
<li>fix(sdk-trace-base): avoid a Webpack self-reference error in
CommonJS output <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6981">#6981</a>
<a href="https://github.com/sansynx"><code>@​sansynx</code></a></li>
<li>fix(sdk-metrics): ignore <code>Infinity</code> in exponential
histograms <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7015">#7015</a>
<a href="https://github.com/mwear"><code>@​mwear</code></a></li>
</ul>
<h3>🏠 Internal</h3>
<ul>
<li>perf(sdk-metrics): reuse a single DataView for exponential histogram
bit reads <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6998">#6998</a>
<a href="https://github.com/mwear"><code>@​mwear</code></a></li>
<li>chore(ci): run documentation tests on a weekly schedule <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6920">#6920</a>
<a
href="https://github.com/LarryHu0217"><code>@​LarryHu0217</code></a></li>
<li>feat(ci): support pre-releases and major version bumps in the
release workflow <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6768">#6768</a>
<a
href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a></li>
<li>chore(resources): Ensure that multiple uses of
serviceInstanceIdDetector.detect() return the <em>same</em> value for
<code>service.instance.id</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/0b72a81636fa476e8f1f1afd2ae0c90a1362194c"><code>0b72a81</code></a>
chore: prepare next release (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7044">#7044</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/a9c5338a9f485f1433df9308f24bd7397a4a0321"><code>a9c5338</code></a>
ci: roll prerelease changelog into one final release changelog (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7045">#7045</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/f41805e769ba10fb6dae72a4b7a5a3dc67cca82e"><code>f41805e</code></a>
chore: prepare next release (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7042">#7042</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/b85eb28343ff6234e2bb4d46b7b4a3d360e5ea2f"><code>b85eb28</code></a>
chore(instrumentation-http): fix lint errors (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7039">#7039</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/3f9253009be2ae48419432e79a597ead7be8be6a"><code>3f92530</code></a>
ci: support pre-releases and major version bumps in release workflow (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7035">#7035</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/82a58316a63b6fde62afd268e145b82222d328cf"><code>82a5831</code></a>
docs(otlp-exporter-base): document HTTP exporter options (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6735">#6735</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/e086dec7f9304107ef6d50b5877be88895c06aa7"><code>e086dec</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/59dac70d00d46fa56b2b921cf721fd922730f23d"><code>59dac70</code></a>
chore(deps): update jamesives/github-pages-deploy-action action to
v4.9.0 (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7">#7</a>...</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/d0ce7532b058631ec9eec111c04fefe7fd873e1f"><code>d0ce753</code></a>
chore: add <a
href="https://github.com/maryliag"><code>@​maryliag</code></a> to
maintainers (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7024">#7024</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/03469a129f97265c8eec93d566e9e00d4f741db3"><code>03469a1</code></a>
chore(deps): update open-telemetry/shared-workflows action to v0.10.0
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7032">#7032</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/open-telemetry/opentelemetry-js/compare/v2.10.0...v2.11.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `ts-jest` from 29.4.9 to 29.4.12
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/kulshekhar/ts-jest/releases">ts-jest's
releases</a>.</em></p>
<blockquote>
<h2>v29.4.12</h2>
<p>Please refer to <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v29.4.11</h2>
<p>Please refer to <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v29.4.10</h2>
<p>Please refer to <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">ts-jest's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.11...v29.4.12">29.4.12</a>
(2026-07-22)</h2>
<h3>Features</h3>
<ul>
<li><strong>compiler:</strong> support TypeScript 7 projects through
compatibility aliases (<a
href="https://redirect.github.com/kulshekhar/ts-jest/pull/5386">#5386</a>)</li>
</ul>
<h2><a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.10...v29.4.11">29.4.11</a>
(2026-05-21)</h2>
<h3>Bug Fixes</h3>
<ul>
<li>preserve Bundler on the CJS path under TypeScript &gt;= 6 (<a
href="https://github.com/kulshekhar/ts-jest/commit/39418187515f11b6584d35a4e3ddf50231f74936">3941818</a>),
closes <a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/4198">#4198</a></li>
</ul>
<h2><a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.9...v29.4.10">29.4.10</a>
(2026-05-18)</h2>
<h3>Bug Fixes</h3>
<ul>
<li>pass <code>resolutionMode</code> to
<code>ts.resolveModuleName</code> for hybrid module support (<a
href="https://github.com/kulshekhar/ts-jest/commit/b557a85f85c3fd34523ec3a15293afbdc9dea83c">b557a85</a>)</li>
<li>rebuild <code>Program</code> when consecutive compiles need
different module kinds (<a
href="https://github.com/kulshekhar/ts-jest/commit/a82a2b32c4987a5249fd5284283117dd2fa3be47">a82a2b3</a>),
closes <a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/4774">#4774</a></li>
<li>respect tsconfig <code>moduleResolution</code> instead of forcing
<code>Node10</code> (<a
href="https://github.com/kulshekhar/ts-jest/commit/1bffffc667557c173ae0c1f93dd436920775dac4">1bffffc</a>)</li>
<li><strong>transformer:</strong> transpile <code>mjs</code> files from
<code>node_modules</code> for CJS mode (<a
href="https://github.com/kulshekhar/ts-jest/commit/96d025dd912ea2bceb18b67d2d509ada7a756d9d">96d025d</a>)</li>
<li><strong>transformer:</strong> use a consistent comparator in
hoist-jest sortStatements (<a
href="https://github.com/kulshekhar/ts-jest/commit/8a8fd2fb8446655bba18367db9306a1089490e62">8a8fd2f</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/3f05625da10da954fdf0a10394385008275ddbb3"><code>3f05625</code></a>
chore(release): 29.4.12</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/df28b27f2e60edf275866763a3cdf745360d3eae"><code>df28b27</code></a>
docs: clarify TypeScript version prerequisites</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/c8a614af419b1dcc90f3d1a7a48238ac1b637e6b"><code>c8a614a</code></a>
docs: mention TypeScript 7 setup in README</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/06c79d4cebfa785749b2f96ef2dbeffc12798c47"><code>06c79d4</code></a>
fix: address TypeScript 7 review feedback</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/f10746008e512203ad7d8c581e2c58cc7dcd43c8"><code>f107460</code></a>
docs: explain TypeScript 7 compatibility setup</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/33882274c1e4fc3a06cece3b21f4873182c8fee7"><code>3388227</code></a>
test(e2e): add TypeScript compatibility matrix</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/891dc731ff9f31785ad9698e4d7cfa6078991fe6"><code>891dc73</code></a>
fix(compiler): support TypeScript 7 compatibility aliases</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/eb135ebe819991b1e10c998915cc6db2057c4de1"><code>eb135eb</code></a>
build(deps-dev): bump shell-quote from 1.8.4 to 1.10.0 in /examples</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/d5d80a34bc575be130db2b9f5cd0958173982cda"><code>d5d80a3</code></a>
ci: pin google osv scan action at v2.3.5</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/6bf293f0a4ddf468735d81fcdf04f923278e030c"><code>6bf293f</code></a>
build(deps): bump shell-quote from 1.8.4 to 1.10.0 in /website</li>
<li>Additional commits viewable in <a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.9...v29.4.12">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-09-06 09:41:24 -07:00
Lance Release c7980dbc40 Bump version: 0.39.0-beta.2 → 0.39.0-beta.3 2026-09-06 08:59:56 +00:00
Jack Ye 1b0f9329ea fix: compare Function output list children by type only (#4137) 
A table whose Function output type contains a list cannot be appended
to. That is every embedding column.

`add()` re-validates the table's own schema against its bindings before
it looks at the incoming data, so the failure does not depend on what
you are writing:

```
ValueError: Invalid input, Function output 'udf_text_embedding_384'
            type no longer matches binding 'fb_...'
```

The check required a list child to be identical to the declaration.
Lance rewrites a list item's name and nullability when it writes, so a
declared `fixed_size_list<item: float not null>` is stored as
`fixed_size_list<item: float>` and never matches again.

The server already draws this distinction —
`job_executor::function_arrow_type::equivalent` compares list children
by type and struct children by identity — which is why declaring the
column succeeded in the first place. This brings the client's copy of
the check into line so the two agree on what a valid Function column
looks like.

Struct children still compare by name and nullability, and the list
length is still part of the declaration.
 2026-09-06 01:58:40 -07:00
Lance Release e5cc7a4d66 Bump version: 0.39.0-beta.1 → 0.39.0-beta.2 2026-09-05 23:26:00 +00:00
Jack Ye 21f11b4463 feat!: replace get_job/job_history with describe_job/query_job_events (#4130) 
A 1M-row column refresh over 200 fragments produced no visible result,
and the client could only ever say `"running"`. Everything needed to
diagnose it already existed server-side — the job registry records a
`claim`/`claim_complete` pair per fragment carrying `rows_processed` —
but none of it was reachable.

## Before

Four ways to ask about a job, none of which told you much.

```python
job = table.refresh_column_async("embedding")
job.status()               # "running". That was the entire debug surface.
db.get_job(job_id)         # state, and a spec. No result, no progress.
db.job_history(job_id)     # raw record batches, no limit, no filter
db.job(job_id)             # a handle that knew nothing
```

## After

Open a job the way you open a table; the handle answers everything.

```python
job = db.open_job(job_id)      # raises JobNotFoundError if there is no such job
```

```python
>>> print(job)
Job(
    id='job-1',
    state='failed',
    job_type='refresh_column',
    creation_ms=1757000000000,
    spec={
        "column": "embedding",
        "num_workers": 4
    },
    failure=JobFailureInfo(phase='execute', message='worker died', retryable=True),
)
```

Individual fields are there too — `job.state`, `job.job_type`,
`job.creation_ms`, `job.spec`, `job.result`, `job.failure` — and
`job.result` carries `rows_assigned` / `rows_failed` as soon as the job
succeeds, with no `wait()` required.

Per-fragment progress *while it is still running*:

```python
done = job.events(filter="state = 'claim_complete'", limit=10_000)
done.column("rows_processed").to_pylist()      # [5000, 5000, ...]
```

The handle an async action returns is the same object, one `refresh()`
away:

```python
job = table.refresh_column_async("embedding")
job.refresh()
job.state, job.result
```

TypeScript is the same experience, down to `console.log`:

```ts
const job = await db.openJob(jobId);   // rejects if there is no such job
console.log(job);                      // same multi-line layout
job.state; job.jobType; job.spec; job.result; job.failure;
const done = await job.events({ filter: "state = 'claim_complete'", limit: 10_000 });
```

## Why each piece matters

- **A result without waiting.** `rows_assigned` / `rows_failed` used to
live only on the terminal result, so a job that never terminated
reported nothing at all.
- **`limit`.** The server caps event rows at 1000 and truncates without
saying so, which silently hid most of a 200-fragment job's history.
- **`filter`.** `claim_complete` rows carry per-claim `rows_processed` —
the only progress signal that exists mid-flight.
- **Events outlive the worker.** They live in the job registry, not in
pod logs that vanish with the pod.
- **One place to ask.** `open_job` replaces `describe_job`,
`query_job_events` and `job`, so a question about a job has one answer
instead of one per calling location.
- **A missing job is an error, not a `None`.** The common case is a job
id copied out of a log, where absence is the surprise worth raising —
and it matches `open_table`.
- **Printing is the debug surface.** Every field on its own line, JSON
payloads keeping their structure. An unrefreshed handle stays on one
line, because there is nothing to lay out.
- **In-process jobs say so.** A local refresh reports `state` and leaves
the rest null rather than inventing fields it has no record for.

`list_jobs` and `cancel_job` stay as they were: one lists, the other is
a one-shot action that should not need a describe first.

## Breaking

All shipped in 0.38.0. No deprecated aliases.

| Was | Now |
| --- | --- |
| `Connection.get_job` → `describe_job` | `Connection.open_job` returns
a populated `Job`, or raises |
| `Connection.job_history` → `query_job_events` | `job.events(...)` |
| `Connection.job` | `Connection.open_job` |
| Python events → `List[pa.RecordBatch]` | `pa.Table` |
| `JobDescription.spec_json` / `.result_json` | internal; use `job.spec`
/ `job.result` |

Node's `Job` is now a TypeScript class wrapping the native handle, so it
returns an Arrow table and parsed values like Python does. New
`Error::JobNotFound` / `JobNotFoundError`; the three job exceptions are
now in the Python API reference.
 2026-09-05 16:24:23 -07:00
Drew 8c9c5c5a5f fix: stop enabling stable row ids on blob table create (#4126) 
This PR stops blob table create from implicitly enabling stable row ids.
A blob schema still selects Lance file format 2.2, but row id behavior
stays with the table config.

Compact then fetch with a `_rowid` captured before compaction is still
not supported on a default table. That needs `take` to remap row
addresses through blob reuse rather than making stable row ids a
blob-table default.

BREAKING CHANGE: blob create no longer enables stable row ids. A blob
schemastill selects Lance file format 2.2. Fetch uses `_rowid` on HEAD.
Held ids survive compact only when the table has stable row ids.


## Testing

* `cargo fmt --all`
* `ruff format .`
* `ruff check .`
* `cargo clippy --quiet --features remote --tests --examples -p lancedb`
* `cargo test --quiet --features remote -p lancedb --test
blob_integration`
* `python/.venv/bin/pytest python/python/tests/test_blob.py -q`
 2026-09-04 14:52:15 +08:00
Jack Ye aab23eb39e feat: support nullable named function outputs (#4123) 
Supports fully nullable named Function outputs while preserving the
distinction between a valid all-null struct and a null/unassigned
result.

## Concrete example

This UDF contract is now valid:

```python
@udf(
    input_schema=pa.schema([
        pa.field("text", pa.string(), nullable=False),
    ]),
    output_schema=pa.schema([
        pa.field(
            "embedding",
            pa.list_(pa.float32(), list_size=1024),
            nullable=True,
        ),
        pa.field("embedding_failure_reason", pa.string(), nullable=True),
        pa.field("embedding_failure_code", pa.int32(), nullable=True),
    ]),
)
def embed(text):
    ...
```

A successful row can return:

```text
embedding = [0.12, ...]
embedding_failure_reason = NULL
embedding_failure_code = NULL
```

If remote inference still fails after retries, it can return:

```text
embedding = NULL
embedding_failure_reason = "HTTP 429: rate limited"
embedding_failure_code = 429
```

An all-null but valid result struct is also assigned; it is not mistaken
for unfinished work.

## Binding shapes

- Mapping the result to one output column stores the `StructArray`
directly, including its parent validity bitmap.
- Flattening the result into top-level columns stores the parent
validity in a reserved internal nullable Boolean assignment column that
is not part of the UDF result mapping.
- An outer null struct remains unassigned/skipped. A valid struct
remains assigned regardless of which child fields are null.
- Scalar Function outputs remain non-nullable.

The contract is preserved through Python registration, Rust application
planning, persisted `FunctionBinding` metadata, schema revalidation, and
Enterprise execution.
 2026-09-03 22:23:53 -07:00
Jack Ye e639b1b650 feat: add asynchronous remote SQL queries (#4070) 
## Summary

Add SQL execution to remote LanceDB connections. On the standard
synchronous connection, `execute_query` waits for the initial result
stream and returns its Arrow reader. `execute_query_async` is called
without Python `await` and immediately returns a query handle for status
inspection, streaming, or cancellation. Local databases report that SQL
is not supported.

The transport and query lifecycle live in Rust. Python exposes
native-backed synchronous and asynchronous connection methods and query
wrappers; it does not use PyArrow's Flight client.

## User experience

The standard synchronous connection supports both direct reads and
background query execution:

```python
db = lancedb.connect(
    "db://analytics",
    api_key="ldb_...",
    sql_host_override="grpc+tls://sql.example.com:10026",
)

# Direct execution waits only until the initial result stream is available.
# Later batches continue streaming as the query progresses.
reader = db.execute_query(
    "SELECT * FROM events",
    default_namespace_path=["production"],
)
for batch in reader:
    print(batch.num_rows)

# Background execution returns a query handle immediately. Despite the
# `_async` suffix, no Python `await` is needed on a synchronous connection.
query = db.execute_query_async("SELECT * FROM events")
print(query.id)

description = db.describe_query(query.id)
print(description.status)
print(description.progress)
print(description.expires_at)

# Start reading as soon as the service advertises partial results. The reader
# continues polling and yields newly available record batches until the query
# and all result endpoints are complete.
reader = query.reader()
for batch in reader:
    print(batch.num_rows)

# Or cancel a different still-running query. Its status becomes "cancelling"
# while the server is still working, then "cancelled" once confirmed.
cancelled_query = db.execute_query_async("SELECT * FROM large_events")
cancelled_query.cancel()
```

The less commonly used asynchronous connection exposes the same
operations as coroutines:

```python
async_db = await lancedb.connect_async(
    "db://analytics",
    api_key="ldb_...",
    sql_host_override="grpc+tls://sql.example.com:10026",
)
query = await async_db.execute_query_async("SELECT * FROM events")
async for batch in await query.reader():
    print(batch.num_rows)
```

The UUIDv7 query id is scoped to the connection that submitted it. The
connection retains lightweight shared query state used by
`query.describe()` and `db.describe_query(query.id)`; the id does not
encode SQL or a Flight continuation token and is not a cross-connection
resume token. Abandoned state has bounded retention, and terminal state
remains available briefly.

Unqualified table names use the connected database and the `public`
namespace by default. `default_namespace_path` accepts a list such as
`["production", "events"]`. SQL can still use qualified names to
reference other databases and namespaces available to the deployment.

## Design

- Uses Arrow Flight `PollFlightInfo` for submission and long polling,
`DoGet` for results, and `CancelFlightInfo` for cancellation. Each
`PollInfo.info` is treated as the cumulative set of currently available
endpoints, so advertised tickets are consumed once and batches can be
delivered before execution is complete.
- Serializes result completion and cancellation into one lifecycle. A
server-accepted request reports `cancelling` and wakes blocked
status/result work; a later retry can confirm `cancelled`. Result
retrieval is rejected after cancellation is accepted, while cancellation
after a result was already delivered is a no-op.
- Assigns a time-ordered UUIDv7 connection-scoped query id and retains
only shared evolving lifecycle state, keeping SQL, Flight continuation
tokens, and Arrow result data out of public ids and the registry.
- Leaves admission control to the server while honoring server
expiration and a local fallback retention window for abandoned entries.
- Retains terminal ids for five minutes so they remain available for
connection-level description.
- Keeps one lazily initialized SQL client on each remote database
connection and attaches fresh authentication, routing, namespace, and
request metadata to every operation.
- Applies the configured overall timeout to each execution, description,
reader, and cancellation operation. A result reader carries one absolute
deadline from `reader()` through the end of streaming; connect and read
timeouts continue to bound their individual phases.
- Returns a bounded, backpressured, single-consumer Arrow stream rather
than collecting the full result in memory. Dropping the reader stops
downloading but does not implicitly cancel the server query.
- Preserves typed schemas for empty result sets through the stream
schema.
- Accepts Flight result messages up to 1 GiB so a valid row containing a
large blob, string, or vector is not rejected by tonic's 4 MiB default
receive limit.
- Supports the Python client first while keeping the authoritative
implementation in the Rust core.
 2026-09-03 14:59:14 -07:00
Will JonesClaude Opus 5Xuanwo
 		2779b75d0d fix(node): resolve remaining pnpm audit findings (#4073) 
`pnpm audit` in `nodejs/` reported a number of vulnerable transitive
dependencies. Most were resolved by `pnpm audit --fix`, which bumped the
affected packages in the lockfile; the `minimumReleaseAgeExclude`
additions in `pnpm-workspace.yaml` are its bookkeeping, exempting the
specific patched versions from the repository's 24-hour hold on newly
published packages. Two findings needed handling by hand, because the
vulnerable package could not simply be moved to a newer release in
place.

`@opentelemetry/sdk-metrics` 1.30.1 pins `@opentelemetry/core` to its
own exact version, and the 1.x line is end-of-life, so
GHSA-8988-4f7v-96qf (unbounded memory allocation in W3C Baggage
propagation) has no fix available on 1.x. This PR moves the dependency
to 2.x, which brings in a patched `@opentelemetry/core`. It is a
dev-only dependency with a single consumer, `__test__/otel.test.ts`, and
the parts of the API that test uses are unchanged between 1.x and 2.x.

`@huggingface/transformers` pins `sharp: ^0.33.5`, and no released
version of transformers has moved past `^0.34.5` — every version in
those ranges inherits the libvips CVEs in GHSA-f88m-g3jw-g9cj, so there
is no upstream release to upgrade to. This PR adds a pnpm `overrides`
entry pinning sharp to the patched `^0.35.4` line instead.

`pnpm audit` now reports no known vulnerabilities.

## Not included

The sharp override only applies to this repository's own dependency
tree, since pnpm overrides are not published to npm. Anyone installing
`@lancedb/lancedb` together with the optional
`@huggingface/transformers` still resolves sharp 0.33.5, and will until
transformers itself moves to sharp 0.35. Practical exposure there is
low: the CVEs require decoding untrusted images, and LanceDB's
transformers embedding function is text-only.

`nodejs/examples/` is a separate install with its own lockfile and is
untouched here. It pins `sharp: "0.33.5"` directly and `pnpm audit`
reports 19 findings against it. Bumping sharp there is more involved
than it looks, because sharp 0.35 requires Node >= 20.9 while the
examples tests run on the Node 18/20 CI matrix, so it is left for
separate work.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-09-03 07:31:03 +08:00
lancedb-gatefixer[bot] 0d19a6c546 fix: support nested-list FTS indexing (#4059) 
## Summary

- validate native FTS fields against the recursively resolved terminal
text leaf
- preserve canonical public paths and list depth for Lance
document-boundary handling
- cover async nested-list index creation and deepest-list `_doc_index`
search coordinates

## Root cause

The FTS resolver recursively found the terminal text field but returned
the outer list field. Native validation therefore rejected
`List(List(Utf8))` before Lance could create a list-element index.

## Validation

- `cargo test --quiet --features remote -p lancedb
test_nested_list_fts_uses_deepest_document_coordinates -- --nocapture`
- `cargo test --quiet --features remote -p lancedb
test_execute_async_validates_fts_input_before_starting_job`
- `cargo test --quiet --features remote -p lancedb
test_public_fts_field_path_prefers_exact_case`
- `cargo check --quiet --features remote --tests --examples`
- `cargo clippy --quiet --features remote --tests --examples`
- `cargo fmt --all -- --check`

Fixes #4058

<!-- lance-gatekeeper-fix:v1 agent=a4e48f49d12b2b6de7f10adb1c5bf5f5
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-09-02 17:51:21 +08:00
Lance Release c0f33f8627 Bump version: 0.39.0-beta.0 → 0.39.0-beta.1 2026-09-02 05:28:44 +00:00
LanceDB Robot 904bd975e5 chore: update lance dependency to v12.0.0-beta.11 (#4118) 
Updates the Rust workspace Lance dependencies and Java lance-core
dependency to v12.0.0-beta.11.

No compatibility fixes were required. Trigger:
https://github.com/lance-format/lance/releases/tag/v12.0.0-beta.11
 2026-09-01 22:27:12 -07:00
Will JonesandClaude Opus 5 d2ca0ce0ab feat: accept multiple on columns for merge insert on remote tables (#4102) 
Merge insert has always taken a list of columns to match on, and local
tables have always joined on all of them. Remote tables did not: any
list longer than one was rejected with `MergeInsertBuilder only supports
a single 'on' column`, so a composite-key upsert was impossible against
LanceDB Cloud and Enterprise from Rust, Python or TypeScript.

The remote request now carries `on` as a list and sends it as one
repeated query parameter per column — `?on=shard_key&on=id`. That is how
the lance-namespace spec encodes an array-valued `on`, so the server
receives a composite key in the shape it expects. A single column still
serializes to `?on=id`, exactly what clients sent before, so existing
callers are unaffected. A column repeated within `on` is now rejected
client-side rather than sent for the server to reject with a 400.

No binding changes were needed: `Table.merge_insert` in Python and
`Table.mergeInsert` in TypeScript already accepted a list, it just could
not reach a remote table. Both gain a test for composite keys, and the
doc comments now say what passing several columns means.

Part of
[ENT-2084](https://linear.app/lancedb/issue/ENT-2084/mergeinsertintotablerequest-support-multiple-columns-for-the).

## Example

```python
table.merge_insert(["shard_key", "id"]) \
    .when_matched_update_all() \
    .when_not_matched_insert_all() \
    .execute(new_data)
```

A row whose `id` matches an existing row but whose `shard_key` differs
is an insert, not an update.

## Not included

Java. Java callers reach merge insert through
`org.lance.namespace.LanceNamespace`, whose
`MergeInsertIntoTableRequest.on` is a single string until
lance-namespace 0.12
([lance-namespace#363](https://github.com/lance-format/lance-namespace/pull/363),
[lance#8915](https://github.com/lance-format/lance/pull/8915)). There is
nothing in this repo's Java SDK to change until the `lance-core` pin can
move.

Sending more than one column requires a server that accepts the repeated
parameter ([sophon#7571](https://github.com/lancedb/sophon/pull/7571));
an older server returns a 400 rather than silently merging on one
column.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-01 16:32:04 -07:00
Bruno Ramirez 9a1ffb9e02 fix(remote): forward create index replace flag (#4115) 
Remote create-index requests already expose `replace` on the builder,
but the remote client did not consistently forward an explicit
`replace=false` over REST. That meant create-only intent could be lost
before it reached a remote server, even though local builders and Python
APIs can express it. This PR forwards `replace=false` on the existing
`create_index` endpoint and keeps the current default behavior unchanged
for compatibility.

This was accomplished with the following changes:

- Serialize `replace: false` into the existing remote create-index
request body when the builder is configured with `.replace(false)`.
- Forward `replace` through the synchronous Python remote `create_index`
wrapper so `RemoteTable.create_index(..., replace=False)` reaches the
repaired path.
- Continue omitting `replace` for the default path so existing remote
create-index requests keep their current semantics.
- Document `name` and `replace` on the existing OpenAPI create-index
request schema.
- Add coverage that verifies the remote client uses the existing
`/create_index/` route and forwards `replace=false`, including the
synchronous Python unified API.

### Testing

- `cargo fmt --all --check`
- `cargo test -p lancedb --features remote
test_create_index_forwards_replace_false_on_existing_route --locked`
- `uv tool run maturin develop --extras tests,dev,embeddings`
- `uv run --frozen pytest
python/tests/test_remote_db.py::test_remote_create_index_new_api`
- `uv run ruff format --check python/lancedb/remote/table.py
python/tests/test_remote_db.py`
- `cargo build -p lancedb --features remote --locked`
- `cargo clippy -p lancedb --features remote --all-targets --locked --
-D warnings`
 2026-09-01 11:54:53 -07:00
LanceDB Robot f2eb4a245d chore: update lance dependency to v12.0.0-beta.9 (#4116) 
Updates Lance dependencies from v12.0.0-beta.5 to v12.0.0-beta.9 across
Rust and Java. No compatibility fixes were required; full workspace
Clippy passes with all features.

Lance tag:
https://github.com/lance-format/lance/releases/tag/v12.0.0-beta.9
 2026-09-02 00:25:27 +08:00
Xuanwo e6867f7d04 feat: support nested blob function signatures (#4109) 
Function signatures currently reject Blob v2 fields nested inside
structs, preventing UDFs from accepting or returning structured values
that contain blobs.

Accept canonical Blob v2 fields as direct or recursive struct children
while preserving exact field metadata and nullability. Blob fields under
list, large-list, fixed-size-list, or map ancestors remain rejected
because collection runtime adaptation is outside the supported Function
ABI.

A whole named struct result can bind directly to one destination column
without introducing an extra wrapper level.
 2026-09-01 23:51:08 +08:00
Xuanwo 193c5e3458 feat: add list_functions client APIs (#4108) 
Function registration and exact lookup are exposed through the SDK, but
clients cannot discover published versions even though the server
provides `POST /v1/functions/list`.

Add Rust and Python sync/async `list_functions()` APIs that return typed
`FunctionVersion` values. The remote client requests canonical
definitions and follows opaque page tokens until the listing is
complete, including empty intermediate pages, while preserving the
server's name/version ordering. Local databases retain the existing
Function-catalog unsupported error.

The SDK consumes protocol pagination internally so callers receive the
complete catalog rather than handling server-specific page tokens.
 2026-09-01 23:50:57 +08:00
Lance Release 7ebd3c222d Bump version: 0.38.0 → 0.39.0-beta.0 2026-09-01 13:16:03 +00:00