rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-10-13 07:12:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,071 Commits 374 Branches 556 Tags
00f5dd6f77e6a6a2387a4bae613f2fed35ca4525
Commit Graph
3071 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
lancedb automation 00f5dd6f77 chore: update lance dependency to v13.1.0-beta.2 2026-10-02 13:06:25 +00:00
dependabot[bot] f72779ebb5 build(deps): bump the rust-minor-patch group across 1 directory with 7 updates (#4355) 
Bumps the rust-minor-patch group with 7 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [aws-smithy-types](https://github.com/smithy-lang/smithy-rs) | `1.6.3`
| `1.6.4` |
| [object_store](https://github.com/apache/arrow-rs-object-store) |
`0.14.1` | `0.14.2` |
| [serde_with](https://github.com/jonasbb/serde_with) | `3.23.0` |
`3.24.0` |
| [napi](https://github.com/napi-rs/napi-rs) | `3.12.2` | `3.12.5` |
| [napi-derive](https://github.com/napi-rs/napi-rs) | `3.6.5` | `3.6.9`
|
| [napi-build](https://github.com/napi-rs/napi-rs) | `2.4.2` | `2.5.0` |
| [arc-swap](https://github.com/vorner/arc-swap) | `1.9.1` | `1.9.2` |


Updates `aws-smithy-types` from 1.6.3 to 1.6.4
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/smithy-lang/smithy-rs/commits">compare
view</a></li>
</ul>
</details>
<br />

Updates `object_store` from 0.14.1 to 0.14.2
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs-object-store/releases">object_store's
releases</a>.</em></p>
<blockquote>
<h2>object_store v0.14.2</h2>
<!-- raw HTML omitted -->
<h1>Changelog</h1>
<h2><a
href="https://github.com/apache/arrow-rs-object-store/tree/v0.14.2">v0.14.2</a>
(2026-09-11)</h2>
<p><a
href="https://github.com/apache/arrow-rs-object-store/compare/v0.14.1...v0.14.2">Full
Changelog</a></p>
<p><strong>Implemented enhancements:</strong></p>
<ul>
<li>Implement <code>Clone</code> for <code>MicrosoftAzure</code> <a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/824">#824</a></li>
<li>Support URLs for jurisdiction-restricted buckets on R2 <a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/814">#814</a></li>
<li>Lightweight DNS cache with shuffle to prevent DNS flooding <a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/726">#726</a></li>
<li>Allow presigning object_store multipart uploads <a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/271">#271</a></li>
<li>Allow specifying <code>Content-Type</code> when signing object_store
PUT requests <a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/270">#270</a></li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Incomplete ASF license header in <code>src/client/s3.rs</code>
(caused -1 vote on 0.14.2 RC1) <a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/853">#853</a></li>
<li>WriteMultipart::finish should abort after part upload failure <a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/818">#818</a></li>
<li>The 1.85 MSRV is inaccurate with all features <a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/811">#811</a></li>
<li>When the backend service returns an HTTP 500, the object store
panics <a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/414">#414</a></li>
</ul>
<p><strong>Documentation updates:</strong></p>
<ul>
<li>Docs: point users uploading large objects at the multipart API <a
href="https://redirect.github.com/apache/arrow-rs-object-store/pull/839">#839</a>
(<a href="https://github.com/alamb">alamb</a>)</li>
<li>[object-store]: update release schedule <a
href="https://redirect.github.com/apache/arrow-rs-object-store/pull/834">#834</a>
(<a href="https://github.com/alamb">alamb</a>)</li>
<li>Add doc example for multipart upload to
<code>GoogleCloudStorage::create_multipart</code> <a
href="https://redirect.github.com/apache/arrow-rs-object-store/pull/803">#803</a>
(<a href="https://github.com/alamb">alamb</a>)</li>
<li>Add doc example for multipart upload to
<code>MicrosoftAzure::create_multipart</code> <a
href="https://redirect.github.com/apache/arrow-rs-object-store/pull/802">#802</a>
(<a href="https://github.com/alamb">alamb</a>)</li>
<li>Add doc example for multipart upload to AmazonS3::create_multipart
<a
href="https://redirect.github.com/apache/arrow-rs-object-store/pull/801">#801</a>
(<a href="https://github.com/alamb">alamb</a>)</li>
</ul>
<p><strong>Closed issues:</strong></p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/apache/arrow-rs-object-store/blob/main/CHANGELOG-old.md">object_store's
changelog</a>.</em></p>
<blockquote>
<!-- raw HTML omitted -->
<h1>Historical Changelog</h1>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/apache/arrow-rs-object-store/commit/279572ea60f3a7a6e5237e066a6f4a35eee611e0"><code>279572e</code></a>
Update CHANGELOG for latest main (<a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/855">#855</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs-object-store/commit/4ff7d048073c07bcc1f1dca74436245b2ba20dda"><code>4ff7d04</code></a>
Restore missing first line of ASF license header in src/client/s3.rs (<a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/854">#854</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs-object-store/commit/25fa079c41fecb8011d5b8c3019f20e09c9b0c74"><code>25fa079</code></a>
Update version to 0.14.2 and add CHANGELOG (<a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/850">#850</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs-object-store/commit/c37db72b92b78608291c1b33feb79bb5bb6d2d1c"><code>c37db72</code></a>
feat: retry failed multipart part uploads (<a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/849">#849</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs-object-store/commit/135984fdf91f813c48c47b6288586d9b0e205164"><code>135984f</code></a>
Add doc hints pointing large-object uploads at the multipart API (<a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/839">#839</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs-object-store/commit/b07471e2bc341278f86e30cf80a850d56cbe2c67"><code>b07471e</code></a>
[object-store]: update release schedule (<a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/834">#834</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs-object-store/commit/978fb48b9ea6ebe3f8f4833f656e793cde79ab15"><code>978fb48</code></a>
Fix MSRV violation in <code>parse.rs</code> test, check
<code>--all-targets</code> in MSRV CI (<a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/837">#837</a>)</li>
<li><a
href="https://github.com/apache/arrow-rs-object-store/commit/e2a49349622aa7ad5db12b107a557d957ac01241"><code>e2a4934</code></a>
Speed up MSRV CI check by installing cargo-msrv via
taiki-e/install-action (#...</li>
<li><a
href="https://github.com/apache/arrow-rs-object-store/commit/e28807de7338abebf14ecf2ab959294cb3096d83"><code>e28807d</code></a>
feat: presigned URLs with extra query params and signed headers
(SignedUrlOpt...</li>
<li><a
href="https://github.com/apache/arrow-rs-object-store/commit/5c743637e97d2f3417b6fd54532bb1223c6547c1"><code>5c74363</code></a>
Restore UnwindSafe/RefUnwindSafe on ClientOptions by bounding
DnsResolver (<a
href="https://redirect.github.com/apache/arrow-rs-object-store/issues/836">#836</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/apache/arrow-rs-object-store/compare/v0.14.1...v0.14.2">compare
view</a></li>
</ul>
</details>
<br />

Updates `serde_with` from 3.23.0 to 3.24.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/jonasbb/serde_with/releases">serde_with's
releases</a>.</em></p>
<blockquote>
<h2>serde_with v3.24.0</h2>
<h3>Added</h3>
<ul>
<li>Implement <code>JsonSchemaAs</code> for
<code>NoneAsEmptyString</code>, so fields using it compile with the
<code>schemars_*</code> features (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1014">#1014</a>)</li>
</ul>
<h3>Changed</h3>
<ul>
<li>Make <code>with_prefix!</code> and <code>with_suffix!</code>
available with the <code>alloc</code> feature, instead of only with
<code>std</code> (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1008">#1008</a>)</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>Handle potential panics in deserialization of chrono and time
DateTimes.
This was privately reported by <a
href="https://github.com/NotAFlightRisk"><code>@​NotAFlightRisk</code></a>.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/jonasbb/serde_with/commit/35ac84c8faf34d6822991bbf2397056994c9208b"><code>35ac84c</code></a>
Bump version to 3.24 (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1016">#1016</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/7b8e561c709544b417b3c0f05f3c57a753e86417"><code>7b8e561</code></a>
Bump version to 3.24</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/6e576c8d5cc5f31efd26cf783d76eac6607d784e"><code>6e576c8</code></a>
Bump the github-actions group with 2 updates (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1012">#1012</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/7b1814b5cf8f2ce4ef7844105ab26760371af0cf"><code>7b1814b</code></a>
Add schemars support for NoneAsEmptyString (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1014">#1014</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/92eeedd1e5be0f176bd9a0532eb8ab1503396bb2"><code>92eeedd</code></a>
Handle potential panic during deserialization for large datetimes (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1015">#1015</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/3404642aae13a3dfcf4ea0f47749f759b36a85db"><code>3404642</code></a>
Handle potential panic during deserialization for large datetimes</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/c7073b98f150d60c22e1ccdb49f1c7151dd8595f"><code>c7073b9</code></a>
Add schemars support for NoneAsEmptyString</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/6b32290267fa18064894e2fa67bf4bed7fb4bd5f"><code>6b32290</code></a>
Run test with only the alloc feature (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1013">#1013</a>)</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/3361c69817d02b079b15f71d1281f573594cc948"><code>3361c69</code></a>
Run test with only the alloc feature</li>
<li><a
href="https://github.com/jonasbb/serde_with/commit/b43bb81cd8f02f06c2bf4bcd057c83e71442a515"><code>b43bb81</code></a>
Reduce required feature for with_{prefix,suffix} from std to alloc (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/1008">#1008</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/jonasbb/serde_with/compare/v3.23.0...v3.24.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `napi` from 3.12.2 to 3.12.5
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/napi-rs/napi-rs/releases">napi's
releases</a>.</em></p>
<blockquote>
<h2>napi-v3.12.5</h2>
<h3>Fixed</h3>
<ul>
<li>fatal_exception noop in optimized builds (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3509">#3509</a>)</li>
</ul>
<h2>napi-v3.12.4</h2>
<h3>Security</h3>
<ul>
<li><code>Object::unwrap</code> and <code>Object::remove_wrapped</code>
(and the deprecated <code>Env::unwrap</code> /
<code>Env::drop_wrapped</code>) read a <code>TypeId</code> out of a
payload that is not a <code>TaggedObject</code>, giving a heap
out-of-bounds read and a JavaScript-forgeable out-of-bounds write;
payload provenance is now validated before dereferencing. See <a
href="https://github.com/napi-rs/napi-rs/security/advisories/GHSA-rhpj-pggq-896v">GHSA-rhpj-pggq-896v</a>,
fixed in <a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3506">#3506</a>.</li>
<li>The serde deserializer reached <code>unreachable!()</code> on
ordinary JavaScript values and aborted the process instead of returning
an error. See <a
href="https://github.com/napi-rs/napi-rs/security/advisories/GHSA-f334-75xc-qxv3">GHSA-f334-75xc-qxv3</a>,
fixed in <a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3507">#3507</a>.</li>
<li><code>ArrayBuffer::from_external</code>,
<code>TypedArraySlice::from_external</code> and
<code>Uint8ClampedSlice::from_external</code> (plus the compat-mode
<code>Env::create_arraybuffer_with_borrowed_data</code>) built their
slice from freed memory on runtimes without external buffers. See <a
href="https://github.com/napi-rs/napi-rs/security/advisories/GHSA-32mm-r9wp-hrvc">GHSA-32mm-r9wp-hrvc</a>,
fixed in <a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3503">#3503</a>.</li>
</ul>
<h3>Fixed</h3>
<ul>
<li><em>(napi)</em> return errors from the serde deserializer for
unexpected JS value shapes (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3507">#3507</a>)</li>
<li><em>(napi)</em> validate wrapped payload provenance in
Object::unwrap/remove_wrapped (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3506">#3506</a>)</li>
<li><em>(napi)</em> point from_external slices at the engine-owned copy
after finalize reclaims the source (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3503">#3503</a>)</li>
</ul>
<h2>napi-v3.12.3</h2>
<h3>Fixed</h3>
<ul>
<li>CurrentThread waker-stack deadlocks and threadless-wasm Buffer
detachment (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3489">#3489</a>)</li>
<li><em>(napi)</em> catch panics in closure trampolines instead of
aborting (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3473">#3473</a>)</li>
</ul>
<h3>Other</h3>
<ul>
<li><em>(napi)</em> keep anyhow error cause when converting (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3477">#3477</a>)</li>
<li><em>(napi)</em> outline threadsafe callback dispatch (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3465">#3465</a>)</li>
<li><em>(napi)</em> outline class instance construction (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3464">#3464</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/43100baf28a3e5709641e35f892be4da5d62dcb2"><code>43100ba</code></a>
chore: release (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3508">#3508</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/bb7b75a7d53b0757f99813c7fef0537cef044adf"><code>bb7b75a</code></a>
chore(release): publish</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/89d7a0b3636b4454265363dab58df471030b4f85"><code>89d7a0b</code></a>
chore(deps): update dependency oxfmt to ^0.68.0 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3519">#3519</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/55545910c2487da38453e6171330e8fc9133213c"><code>5554591</code></a>
fix(cli): rate-limit failed incarnation probes and let reconciliation
tests i...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/6d10d826c34f5d0009eb1ab47efaa4a2e34f4555"><code>6d10d82</code></a>
feat(cli): make the deferred workerd loader usable by host-driven addons
(<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3520">#3520</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/a1bcecbd1bef2f51f1047f5800d5a312eb2eba3b"><code>a1bcecb</code></a>
feat(cli): add <code>napi.wasm.threadlessInitialMemory</code> for
threadless WASI loader...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/0999babc1e37ce869b79f59db14f9eb12f6647df"><code>0999bab</code></a>
feat(cli): export <code>__napiBindingTarget</code> from generated
binding loaders (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3515">#3515</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/30ea504c3d65e1cb178d3124068257d8df8f3835"><code>30ea504</code></a>
fix(cli): bound symlink-ancestor reconciliation guards to the anchor's
parent...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/ae8f34a8dc5f629b53d136442e8f614f2d3c17b6"><code>ae8f34a</code></a>
fix(cli): run the wasm env cleanup barrier on a raw emnapi
`Context.destroy()...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/ccd314bbb5201182311b74dddd61f60b323318f4"><code>ccd314b</code></a>
fix(cli): never wedge reconciliation locks when host identity probes
fail (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3">#3</a>...</li>
<li>Additional commits viewable in <a
href="https://github.com/napi-rs/napi-rs/compare/napi-v3.12.2...napi-v3.12.5">compare
view</a></li>
</ul>
</details>
<br />

Updates `napi-derive` from 3.6.5 to 3.6.9
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/napi-rs/napi-rs/releases">napi-derive's
releases</a>.</em></p>
<blockquote>
<h2>napi-derive-v3.6.9</h2>
<h3>Other</h3>
<ul>
<li>update Cargo.toml dependencies</li>
</ul>
<h2>napi-derive-v3.6.8</h2>
<h3>Fixed</h3>
<ul>
<li><em>(napi)</em> guard AsyncTask completion against env teardown (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3536">#3536</a>)</li>
</ul>
<h2>napi-derive-v3.6.7</h2>
<h3>Other</h3>
<ul>
<li>update Cargo.toml dependencies</li>
</ul>
<h2>napi-derive-v3.6.6</h2>
<h3>Other</h3>
<ul>
<li>update Cargo.toml dependencies</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/2763e12efc855748485129952a6ccb97ac991c06"><code>2763e12</code></a>
chore: release (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3538">#3538</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/5ae5cf604809ed6b9868c8e67b201bc9eb68d9bf"><code>5ae5cf6</code></a>
chore(release): publish</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/0da138ec836906da5bc3a996521299e1d284bdda"><code>0da138e</code></a>
feat(cli): self-sign OpenHarmony artifacts with --ohos-sign (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3539">#3539</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/2499a6311148492bc01ae316621b514a9345e4e7"><code>2499a63</code></a>
feat(async-runtime): offer the MultiThread flavor on
wasm32-wasip1-threads (#...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/886bd071972e09010ee901d1a1bb669a26d3e0dc"><code>886bd07</code></a>
chore(deps): update dependency oxc-parser to ^0.151.0 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3542">#3542</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/edb0a295a14371b2f943c2a89bacf385eb61a038"><code>edb0a29</code></a>
fix(napi): validate native payload provenance for External and instance
data ...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/38162bb0eb324ae24b402982bad3d4ef3f24c90a"><code>38162bb</code></a>
chore: release (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3529">#3529</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/ec3d8ead66c18e022d2e3fb6832e52e5204ffcc3"><code>ec3d8ea</code></a>
fix(napi): guard AsyncTask completion against env teardown (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3536">#3536</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/96ed2672820b5468b6a451080fc27e6eef594d3d"><code>96ed267</code></a>
chore(deps): update release-plz/action action to v0.5.139 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3537">#3537</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/b2c9f3b610cb167d2c05b8d6c748c120926ffd73"><code>b2c9f3b</code></a>
chore(deps): update release-plz/action action to v0.5.138 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3533">#3533</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/napi-rs/napi-rs/compare/napi-derive-v3.6.5...napi-derive-v3.6.9">compare
view</a></li>
</ul>
</details>
<br />

Updates `napi-build` from 2.4.2 to 2.5.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/napi-rs/napi-rs/releases">napi-build's
releases</a>.</em></p>
<blockquote>
<h2>napi-build-v2.5.0</h2>
<h3>Added</h3>
<ul>
<li><em>(async-runtime)</em> offer the MultiThread flavor on
wasm32-wasip1-threads (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3541">#3541</a>)</li>
</ul>
<h2>napi-build-v2.4.4</h2>
<h3>Fixed</h3>
<ul>
<li><em>(napi)</em> guard AsyncTask completion against env teardown (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3536">#3536</a>)</li>
</ul>
<h2>napi-build-v2.4.3</h2>
<h3>Fixed</h3>
<ul>
<li><em>(cli)</em> drain outstanding async work before disposing a WASI
binding (<a
href="https://redirect.github.com/napi-rs/napi-rs/pull/3528">#3528</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/2763e12efc855748485129952a6ccb97ac991c06"><code>2763e12</code></a>
chore: release (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3538">#3538</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/5ae5cf604809ed6b9868c8e67b201bc9eb68d9bf"><code>5ae5cf6</code></a>
chore(release): publish</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/0da138ec836906da5bc3a996521299e1d284bdda"><code>0da138e</code></a>
feat(cli): self-sign OpenHarmony artifacts with --ohos-sign (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3539">#3539</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/2499a6311148492bc01ae316621b514a9345e4e7"><code>2499a63</code></a>
feat(async-runtime): offer the MultiThread flavor on
wasm32-wasip1-threads (#...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/886bd071972e09010ee901d1a1bb669a26d3e0dc"><code>886bd07</code></a>
chore(deps): update dependency oxc-parser to ^0.151.0 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3542">#3542</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/edb0a295a14371b2f943c2a89bacf385eb61a038"><code>edb0a29</code></a>
fix(napi): validate native payload provenance for External and instance
data ...</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/38162bb0eb324ae24b402982bad3d4ef3f24c90a"><code>38162bb</code></a>
chore: release (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3529">#3529</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/ec3d8ead66c18e022d2e3fb6832e52e5204ffcc3"><code>ec3d8ea</code></a>
fix(napi): guard AsyncTask completion against env teardown (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3536">#3536</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/96ed2672820b5468b6a451080fc27e6eef594d3d"><code>96ed267</code></a>
chore(deps): update release-plz/action action to v0.5.139 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3537">#3537</a>)</li>
<li><a
href="https://github.com/napi-rs/napi-rs/commit/b2c9f3b610cb167d2c05b8d6c748c120926ffd73"><code>b2c9f3b</code></a>
chore(deps): update release-plz/action action to v0.5.138 (<a
href="https://redirect.github.com/napi-rs/napi-rs/issues/3533">#3533</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/napi-rs/napi-rs/compare/napi-build-v2.4.2...napi-build-v2.5.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `arc-swap` from 1.9.1 to 1.9.2
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/vorner/arc-swap/blob/master/CHANGELOG.md">arc-swap's
changelog</a>.</em></p>
<blockquote>
<h1>1.9.2</h1>
<ul>
<li>Document RefCnt must not panic (<a
href="https://redirect.github.com/vorner/arc-swap/issues/208">#208</a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/vorner/arc-swap/commits">compare view</a></li>
</ul>
</details>
<br />

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-10-02 07:49:06 -05:00
XY Zhan 723a2394e5 feat: maintain every index a table has, including ones created later (#4342) 
**The maintained set is an intent, not a snapshot.** An unnamed set is
stored as
the intent to maintain every index, which lance re-resolves whenever it
builds a
MemTable, so an index created after the spec installs is maintained too.
A named
set is checked here, where the caller can be told which names exist; an
index of
a kind the MemWAL cannot mirror fails a named set and is skipped by an
unnamed
one.

**A newly created index reaches a writer that is already open.** lancedb
caches a
`ShardWriter` per shard and reuses it, so `create_index` installs the
current set
on the cached writer. A failure there is reported rather than
propagated: the
index is committed durably by then, and the writer this process holds
may be
fenced or poisoned, which is no reason to fail creating an index.

**A table carries one spec.** `set_lsm_write_spec` fails while one is
installed
and names `unset_lsm_write_spec` in the error, because the generations
already
written were homed under that spec. Accepting a repeat that changed only
the
maintained set would hand a caller who also passed a different sharding
a success
that changed nothing they asked for.

**The read path asks what is actually carried.** An unnamed set leaves
the stored
list empty, so the list alone cannot decide whether an index is
maintained. The
read asks the spec *and* every resident MemTable whether it carries the
index, so
a MemTable built before the index joined the set refuses the read rather
than
answering without its rows. Both spec routes report an unnamed set as
`null`
rather than `[]`: maintain whatever the table gains, and maintain
nothing, are
different settings.

## Known gap

The guard above covers resident MemTables. Once a writer flushes and
closes none
is resident, so a retained generation written before the index existed
still gets
an indexed read that omits its rows.

This window is **wider than on main**: a set resolved at install time
excluded an
index created later, so the read was refused; an intent includes it, and
the
refusal only holds while a MemTable is resident. Closing it needs the
generation's
own coverage, which this layer cannot see — its snapshot carries a
generation and
a path — so it belongs in lance's scanner or flush path.

## Testing

`cargo nextest run -p lancedb --lib --features remote` — **1479 passed,
3
skipped**. `cargo fmt --all --check`, and `cargo clippy --features
remote --lib
--tests --examples -- -D warnings`, both clean.

Covered: the round trip of an unnamed set, a set over an installed spec
refused,
a spec replaced through unset, a read refused while a resident MemTable
predates
the index, and an index created mid-write reaching the open writer.

Two Python binding tests move with the semantics: an unnamed set now
reads back
as `None` rather than the list it resolves to, and the refusal is
matched on
`already set`. Neither was run locally — no Python build here — so CI is
the
check on them.
 2026-10-02 00:11:44 -04:00
LanceDB RobotDaniel RammerClaude Opus 5.5
 		857e455361 chore: update lance dependency to v13.1.0-beta.1 (#4358) 
Updates the Rust workspace and Java Lance dependencies to
[v13.1.0-beta.1](https://github.com/lance-format/lance/releases/tag/v13.1.0-beta.1)
and refreshes Cargo.lock. Removes a redundant `pub(crate)` declaration
in the private Node.js blob module to satisfy the required Clippy check.

Validation: `cargo clippy --quiet --workspace --tests --all-features --
-D warnings`, `cargo fmt --all --quiet`, `pnpm build`, and `pnpm test
__test__/blob.test.ts --runInBand` (26 tests passed) in `nodejs`.

---------

Co-authored-by: Daniel Rammer <hamersaw@protonmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-01 09:13:08 -05:00
Lance Release 4730f9036b Bump version: 0.40.0-beta.11 → 0.40.0-beta.12 2026-09-30 13:32:13 +00:00
Jonathan HsiehandClaude Opus 5.5 82d39485a4 feat(functions): address Functions by namespace path (#4341) 
Adds namespace addressing to Functions, matching what Secrets already
have (#4150).

## API

`create_function` / `create_function_async`, `get_function`,
`list_functions`, `drop_function` / `drop_function_async` take
`namespace_path` — keyword-only in Python (sync, async and remote
connections) and a trailing `&[String]` in Rust — defaulting to the
root. The namespace is addressed in the route identifier
(`/v1/function/analytics$features$embed/describe`,
`/v1/namespace/analytics$features/function/list`); no body carries it.

`FunctionVersion` and `FunctionVersionRef` name their Function **by
parts** — a short `name` plus `namespace_path` — the same shape as
`SecretReference`, so a returned version reopens from its own fields:

```python
fv = db.create_function(embed, namespace_path=["analytics", "features"])
fv.name, fv.namespace_path          # "embed", ("analytics", "features")
db.get_function(fv.name, version=fv.version, namespace_path=list(fv.namespace_path))
```

An empty path is omitted from the wire, so root documents keep their
shape. `namespace_path` is passed straight to the binding, so a bare
string (`namespace_path="prod"`) is a `TypeError` rather than four
one-character segments; segments are checked for addressability before a
request is built.

The binding and application allow-lists now share one field list,
`FUNCTION_VERSION_REF_FIELDS`, which admits `namespace_path` — without
it a namespaced binding or application read as coming from a newer
contract.

## Service side

The Sophon half serves and persists FunctionVersion / FunctionVersionRef
documents in this split form, keeping the joined id as its internal
catalog key. It lands in sophon with the submodule bump.

## Tests

- Rust: every Function verb addresses a namespace in the path; an
unaddressable segment never reaches the transport; `FunctionVersion`
round-trips a namespace and `reference()` carries it; a namespaced
binding and application are known shapes.
- Python: sync and async verbs address the namespace; a created Function
reopens from its own `name`/`namespace_path`; a bare-string path is
refused before any request.

Draft: compiles, `cargo fmt --check` and clippy `-D warnings` clean;
lancedb lib Function tests (116) and `first_class_function_slice2` pass
on this base.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 14:07:14 -07:00
Bilal Atique 0a1c1a04c7 feat(node): accept an AbortSignal to cancel blob reads (#4350) 
Closes #4288.

Blob reads in Node can now be cancelled. `BlobFile.read`, `readRange`
and `readRanges`, and `Table.fetchBlobs` and `fetchBlobFiles` take an
optional `{ signal?: AbortSignal }` (new `BlobReadOptions` type). When
the signal aborts, the call rejects with `signal.reason` (an
`AbortError` unless the caller gave a reason) and the native future is
dropped, so the read stops instead of running to completion. A signal
that is already aborted rejects before any read starts.

How it works:

- On the Rust side, these five methods are now sync `#[napi]` methods
that return `env.spawn_future(...)`. The future is wrapped in
`futures::future::abortable`, and the abort handle is registered with
napi's `AbortSignal::on_abort`. napi's `AbortSignal` is `Rc`-based, so
it can't be an argument of an `async fn`, which is why the methods
changed shape. The TypeScript signatures are unchanged apart from the
new optional argument.
- napi-rs installs its own `onabort` on the signal it receives, which
would overwrite a handler the caller set. So the TypeScript wrapper
passes a private `AbortController` signal to the native call for each
request, forwards the caller's abort to it with `addEventListener(..., {
once: true })`, and removes the listener when the call settles.

Tests in `table.test.ts`:

- an already aborted signal rejects on all five methods, and the handle
still reads afterwards
- a custom abort reason is what the call rejects with
- a read with a signal that never aborts returns normally and leaves no
`abort` listener on the signal
- aborting a 32 MiB `readRange` right after it starts rejects with
`AbortError`, and the same handle reads again afterwards

The Node docs under `docs/src/js` are regenerated.

I didn't add a test for where the `read()` cursor ends up after a
mid-flight abort on a local table. The remote `BlobFile` documents that
a cancelled read leaves the cursor unchanged, but I haven't confirmed
the same for the native Lance handle, so the docs don't claim it.
 2026-09-29 14:01:17 -05:00
lancedb-gatefixer[bot] ad79112b8e fix(node): support LargeUtf8 and LargeBinary Arrow columns (#4287) 
## Cause

The Arrow type sanitizer handled Utf8 and Binary but omitted LargeUtf8
and LargeBinary, so Arrow tables from another Arrow installation failed
schema sanitization before reaching LanceDB.

## Fix

Reconstruct both large Arrow types during sanitization. Regression tests
cover createTable and add with Arrow 15–18 and adding a LargeBinary
Arrow column to a blob table.

## Validation

- Confirmed the new table test failed before the fix with `Unrecognized
type id in schema: 20` on all four Arrow versions.
- The table, sanitizer, and Arrow Jest suites passed: 604 tests.
- `pnpm lint`, `pnpm build`, and `pnpm run docs` passed.

Fixes #4266

<!-- lance-gatekeeper-fix:v1 agent=1090de3c0ddeca587956632b79f0193c
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-09-29 12:25:12 -05:00
lancedb-gatefixer[bot] 85b0eba176 fix(python): support typed embedding vector fields (#3826) 
## Summary

- infer fixed-size float32 Arrow vectors from `VectorField` metadata
when a model uses the standard `list[float]` annotation
- document the mypy-compatible embedding-field declaration while
preserving the existing `Vector(dim)` runtime API
- add regression coverage for schema conversion and model construction

## Root cause

`Vector(dim)` is a runtime constrained-type factory, but Python's static
typing rules do not permit call expressions in annotation positions.
Mypy's suggested `Vector[...]` spelling also cannot represent a
dimension computed at runtime and fails because `Vector` is a function.
`VectorField` already carries the embedding function and its dimension,
but schema conversion previously ignored that metadata for a statically
valid `list[float]` annotation.

The runtime factory remains available for dimension-constrained Pydantic
validation. Embedding-backed models that require mypy compatibility can
now use `list[float] = embedding.VectorField()`; LanceDB derives the
same nullable fixed-size float32 Arrow type from the field metadata.

## Validation

- `cd python && uv run --extra tests pytest
python/tests/test_pydantic.py -q` (23 passed)
- focused mypy check reproducing the original error and accepting the
new `LanceModel` declaration
- isolated schema checks with Pydantic 1.10.24 and 2.11.7
- `ruff format .`
- `ruff check .`

Fixes #1982

<!-- lance-gatekeeper-fix:v1 agent=877facbc659cdc9424bd367e12426366
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-09-29 12:00:52 -05:00
Bilal Atique 4e3ec7cc27 feat(node): expose seek, tell, bounded read, readRanges and close on BlobFile (#4348) 
Closes #4300.

The TypeScript `BlobFile` only had `size()`, `read()` and `readRange()`,
while the Rust `BlobFile` behind it also has `seek`, `tell`,
`read_up_to`, `read_ranges` and `close`. This exposes those on the Node
binding:

- `read(maxBytes?)` reads at most `maxBytes` from the cursor (Rust
`read_up_to`). Without the argument it reads to the end, as before.
- `readRanges([{ start, end }, ...])` reads several half-open ranges and
returns one buffer per range, in order. Like `readRange`, it does not
move the cursor.
- `seek(position)` and `tell()` move and report the cursor.
- `close()` releases the handle, and `isClosed()` reports it. Every
read, `seek` and `tell` after `close()` fails with "blob file is already
closed". A second `close()` does nothing. The binding keeps its own
`closed` flag, the same way the Python `BlobFile` does.

Offsets and lengths are `bigint`, like `readRange` and `size()`.
`BlobRange` is exported as a type.

The new tests in `table.test.ts` cover chunked reads with
`read(maxBytes)`, `seek` and `tell`, a negative seek position,
`readRanges` order and cursor, a `readRanges` end past the blob size,
and reads after `close()`. The Node docs under `docs/src/js` are
regenerated with `pnpm run docs`.

I left the `ReadableStream`/async iterator idea from the issue out of
this PR. The AbortSignal work in #4288 touches the same methods, so it
can follow on top of this.
 2026-09-29 11:32:30 -05:00
Xuanwo ea223c4ec4 chore: update lance dependency to v13.0.0-beta.18 (#4339) 
Updates the Rust workspace dependencies, Cargo lockfile, and Java
`lance-core` to [Lance
v13.0.0-beta.18](https://github.com/lance-format/lance/releases/tag/v13.0.0-beta.18).
It includes lance-format/lance#9574, which bounds the memory an IVF
partition split uses when it re-reads raw vectors during
`optimize_indices`, and lance-format/lance#9582, which keeps the mTLS
REST namespace client on rustls.

Lance's new combined-fields FTS query variant (introduced by
v13.0.0-beta.17) is handled the same way as in #4336: the remote
capability check and Rust/Node bindings accept it, with a regression
test for the remote request payload, and the Python conversion reports
that this query type is not yet supported. This PR supersedes #4336.
 2026-09-29 23:19:59 +08:00
Bilal Atique 166c114710 fix(rust): list indices when an FTS index's files are missing (#4330) 
Fixes #4250.

`list_indices()` reads each FTS index's files to fill `index_details`.
When those files were missing, the error failed the whole call, so the
caller could not list any index, including the damaged one it needs to
rebuild. Scalar indexes were not affected because their listing reads
only the manifest.

Now a failed read logs a warning and leaves that index's `index_details`
as `None`. The other fields still come from the manifest, and
`tokenize()` already returns a clear error when an FTS index has no
details.

The new test `test_list_indices_with_missing_fts_index_files` creates an
FTS index and a BTree index, deletes the FTS index directory, and
reopens the table. It then checks that both indexes are still listed.
Without the change, it fails with the same `Not found` error as the
issue.
 2026-09-29 10:11:34 -05:00
lancedb-gatefixer[bot] 629f39d4a5 fix(python): preserve mixed blob values before Arrow conversion (#4304) 
## Cause
Python list and pandas rows were converted to Arrow before the target
blob v2 schema was applied. In a mixed bytes and URI batch, Arrow
inferred a binary column and encoded the URI as inline bytes; dict and
`lance.Blob` values could fail conversion.

## Fix
Build blob v2 columns from the original Python values using the target
field's storage type before converting the rest of each row. This
preserves inline bytes, external URIs, descriptor dictionaries, blob
objects, and nulls for `add`, `merge_insert`, and `create_table`.

## Validation
- `python/tests/test_blob.py`: 76 passed, 1 skipped.
- Python project Ruff check passed.
- Repository-wide Ruff check still reports 29 existing findings in `ci/`
files outside this change.

Fixes #4271

<!-- lance-gatekeeper-fix:v1 agent=54c95f06f11c88187b07a430aac6be2f
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-09-29 08:38:00 -05:00
lancedb-gatefixer[bot] 51ea02ea8c fix: pin remote BlobFile reads to the opened version (#4295) 
## Cause

On latest-tracking remote tables, `fetch_blob_files` probed a concrete
table version but retained `None` on each handle. Later range reads
resolved the latest version again, so an update or delete could make an
opened handle fail.

## Fix

For nonempty blobs, capture `x-lancedb-version` from the 206 size probe
and include it on every later range request. Reject a nonempty probe
without a valid version. Accept an empty-blob 416 probe with or without
that header, because an empty handle makes no later data range request.
The Python remote mock now covers both response shapes.

## Limitation

A table dropped and recreated with the same name can reuse version
numbers and row addresses. Version pinning does not identify the table
itself, so an old handle can still return data from the recreated table.
That identity case needs a separate fix. Callers that replace a table
should discard its old blob handles.

## Validation

- `cargo test --quiet --features remote -p lancedb --lib
remote::table::blobs::tests` (24 passed)
- Rebuilt the Python extension; `uv run --extra tests pytest
python/tests/test_remote_db.py -q -k remote_blob` (6 passed)
- `cargo clippy --quiet --features remote -p lancedb --lib --tests
--examples`
- `cargo fmt --all -- --check`, `ruff check .`, and `ruff format .`

Fixes #4276

<!-- lance-gatekeeper-fix:v1 agent=e309bdf75b1a87f6492a6dc3ee1bfd81
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-09-29 08:35:19 -05:00
Yang Cen b8af687286 feat(sql): expose composable native vector relations (#4340) 
## Summary

Native vector dedup currently materializes through the native MV refresh
protocol. Add composable DataFusion pair and retained-row relations so a
host can use its existing SQL job and MV writer instead.

`VectorQueryTable` binds a retained snapshot, selects all/first/random
nonempty IVF scopes before dispatch, and exposes `DuplicatePairsExec` as
its physical child. Pair queries can include segment, partition and
snapshot columns and fetch a scalar application key for each endpoint
without reading source vectors for scoring. The default primitive schema
remains unchanged.

`VectorDedupExec` consumes the pair stream, reuses the spillable greedy
direct-representative selection, and scans retained original rows from
that snapshot. A–B–C without A–C retains A and C. This query path has no
durable partition receipts or writes; publication and failure belong to
the enclosing SQL sink. Existing native MV APIs remain available.

`BaseTableAdapter::vector_query_snapshot(None)` captures latest without
changing a shared table handle's pin; `Some(version)` selects a retained
version. A host can retain unbound SQL and resolve latest again for each
refresh. Single-segment random sampling matches Sophon's `ivf_partition`
SplitMix64 policy; multi-segment sampling ranks canonical nonempty
scopes.

Host integration and SQL syntax:
https://github.com/lancedb/sophon/pull/8135. The host authorizes
sources, binds named arguments, distributes the pair child, and uses its
existing SQL Job Registry and atomic MV replacement.

## Validation

All manual validation ran on an AWS r8i.8xlarge VM, not the developer
laptop.

- Focused Rust integration suite: 2 passed, including multiple
segments/partitions, empty results, threshold boundaries, 8,193 rows per
segment, deterministic sampling, mapped application IDs, and
native-oracle pair equality.
- Additional compacted stable/nonstable row-ID and blob test: 1 passed,
including application-key mapping, null vectors, snapshot deletions and
explicit/latest version capture.
- Rust formatting and focused Clippy with warnings denied passed.
- Dependent Sophon SQL-job E2E passes with two worker PEs: pairs,
sampling, direct representatives, latest/fixed snapshot refreshes, and
blob payloads. Killing a SQL worker after partial staging preserves the
old MV; full rerun commits the complete replacement. Performance
measurement is in progress in the dependent PR; no performance claim is
made here.
 2026-09-29 13:24:21 +08:00
lancedb-gatefixer[bot] 8e3606fbe3 fix: handle remote blob fetch request caps (#4309) 
## Cause

Remote `fetch_blobs` sent every row ID in one Cloud request, exceeding
the server's 1024-row or 64 MiB byte cap. A single blob over the byte
cap could not be read through this method.

## Fix

- Send at most 1024 row IDs per request and split batches further when
the server reports the blob-byte cap, preserving input order and nulls.
- Resolve one exact dataset version before multi-request reads. If an
initial single request hits the byte cap, resolve the version before any
successful split or Range read; use that version for every resulting
request.
- Read an individual oversized blob through the existing Range route.
- Document the per-request caps and transparent handling in the Rust,
Python, and TypeScript APIs and generated JS docs.

## Validation

- Focused Rust `fetch_blobs` tests: 26 passed, including row-count,
byte-cap, Range fallback, and exact-version assertions.
- Rust clippy and formatting passed.
- Focused Python blob tests: 13 passed; Ruff lint and formatting passed.
- Node build, lint, docs generation, and focused `fetchBlobs` test
passed.

Fixes #4275

<!-- lance-gatekeeper-fix:v1 agent=6ae94d49bf1d4cae057bba72584e7b90
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-09-28 20:31:36 -05:00
brekenandbreken-ai c119f0c8e5 fix(python): rank each result list separately in RRF multivector rerank (#4325) 
## What's wrong

`RRFReranker.rerank_multivector` concatenates every vector result list
and passes the concatenation to `rerank_hybrid` as a single list.
`rerank_hybrid` then scores each row by its position in that
concatenation, so a row's rank depends on how many rows came before it
in *earlier* lists.

With two searches of 3 rows each (`K=60`):

| row | rank in its own list | score today | RRF score |
|---|---|---|---|
| 1 (top of list 1) | 1 | 1/61 | 1/61 |
| 3 (third of list 1) | 3 | 1/63 | 1/63 |
| 4 (top of list 2) | 1 | **1/64** | 1/61 |
| 5 (second of list 2) | 2 | **1/65** | 1/62 |

So the best hit of the second search ranks below the worst hit of the
first, and the order of the lists passed in decides the fused ranking.
`MRRReranker.rerank_multivector` already scores each list on its own
("MRR semantics require treating each vector result as a separate
ranking system"). RRF is defined the same way.

## Fix

Score each result list with its own ranks (`1 / (rank + K)`, summed
across lists for rows found by more than one), then dedupe the
concatenation by `_rowid`, attach `_relevance_score`, and sort. The
`return_score` handling is unchanged.

## Tests

`test_rrf_multivector_ranks_each_result_list` in
`python/python/tests/test_rerankers.py`:

- On `main`, it fails with `assert 0.015625 == 0.01639344262295082` (the
top hit of list 2 got `1/64` instead of `1/61`).
- With the fix, it passes, and so do the other tests in
`test_rerankers.py` (16 passed, 12 skipped for missing API keys or
models).
- `ruff format --check` and `ruff check` are clean.

The fix was found and written by an AI agent (Breken) and checked
against the tests above.

Co-authored-by: breken-ai <312387581+breken-ai@users.noreply.github.com>
 2026-09-28 20:19:51 -05:00
ningsh7 ef11e44adc feat(node): expose allowExternalBlobOutsideBases in Table.add (#4317) 
## Summary

- Add `allowExternalBlobOutsideBases` to the Node `AddDataOptions` and
forward it through N-API to the Rust `AddDataBuilder`.
- Document its default and local-only external blob behavior.
- Cover URI strings and URI objects, option scoping, progress callbacks,
overwrite mode, and remote rejection.

## Validation

- `pnpm test --runInBand` (21 suites passed, 1 skipped; 908 tests
passed, 6 skipped)
- `pnpm lint-ci`
- `pnpm tsc`
- `pnpm run docs`
- `cargo fmt --all -- --check`
- `CC=clang cargo clippy --profile ci -p lancedb-nodejs --all-features
-- -D warnings`

Fixes #4267
 2026-09-28 20:19:36 -05:00
brekenandbreken-ai 70a1123404 fix(python): write NaN and infinity as SQL literals in table.update (#4326) 
## What's wrong

`value_to_sql` renders floats with `str()`, so `float("nan")` becomes
`nan` and `float("inf")` becomes `inf`. SQL reads those as column names:

```python
table.update(where="id = 1", values={"price": float("nan")})
# ValueError: Invalid input, Schema error: No field named nan. Valid fields are id, price.
```

The same happens for `math.inf`, `-math.inf`, `np.nan` and
`np.float32("inf")`, which go through the same float path. The only
workaround today is `values_sql`.

## Fix

Emit `CAST('NaN' AS DOUBLE)`, `CAST('Infinity' AS DOUBLE)` and
`CAST('-Infinity' AS DOUBLE)` for these values. Finite floats render as
before. The cast also lands correctly in `float32` columns.

## Tests

`test_value_to_sql_nan_and_infinity` in
`python/python/tests/test_util.py` updates `float64` and `float32`
columns to NaN, +inf and -inf, then reads them back:

- On `main`, it fails with `ValueError: Invalid input, Schema error: No
field named nan`.
- With the fix, it passes. All of `test_util.py` passes (75 passed, 1
skipped), and so do the update tests in `test_table.py` (`-k update`, 8
passed).
- `ruff format --check` and `ruff check` are clean.

NaN inside a list value (for example a vector) still fails, because
Lance only accepts literals inside array values. This PR covers scalar
columns only.

The fix was found and written by an AI agent (Breken) and checked
against the tests above.

Co-authored-by: breken-ai <312387581+breken-ai@users.noreply.github.com>
 2026-09-28 20:16:26 -05:00
Mohammad Hijjawi 580240278f fix(python): apply the default limit of 10 to sync hybrid queries (#4344) 
## Summary

A synchronous hybrid query without an explicit `.limit()` can return up
to 20 rows. The documented default is 10, and `AsyncHybridQuery` already
falls back to `DEFAULT_HYBRID_LIMIT` (10).

In `LanceHybridQueryBuilder`, each sub-query (vector and FTS) falls back
to its own default of 10 rows. The fused result was then sliced with
`limit=None`, i.e. not truncated at all. When the two legs match
different rows, the caller gets the union.

t = db.create_table("t", [{"text": ("dog" if i >= 20 else "cat") + f"
{i}", "vector": [float(i), 0.0]} for i in range(40)])
    t.create_index("text", config=FTS())
len(t.search(query_type="hybrid").vector([0.0,
0.0]).text("dog").to_arrow())
    # before: 20, after: 10

For the same reason, `.offset(n)` without `.limit()` did not raise the
sub-query limits to cover the skipped prefix; that only happened inside
`if self._limit:`.

## Changes

- Use `DEFAULT_HYBRID_LIMIT` when no limit is set, both for the
sub-query limits (`limit + offset`) and for the final slice of the
reranked results, matching `AsyncHybridQuery`.

## Test plan

- [x] New `test_hybrid_query_default_limit` in `test_hybrid_query.py`.
It is the sync counterpart of `test_async_hybrid_query_default_limit`,
plus an `offset()`-without-`limit()` check. It fails before the fix (20
rows) and passes after.
- [x] `pytest python/tests/test_hybrid_query.py
python/tests/test_rerankers.py`
- [x] `ruff format --check` / `ruff check`

I have no local Rust toolchain, so I ran the tests against the released
0.39.0 native module with the changed Python files overlaid. The changed
functions are identical between 0.39.0 and main.
 2026-09-28 20:12:00 -05:00
Jay ZhouJay ZhouClaude Opus 5.5
 		1a463f3633 fix(python): warn when optimize cleans up all old versions (#4337) 
## Why

`optimize(cleanup_older_than=timedelta(0))` deletes the data files of
every version except the latest. Any other handle still on an older
version then fails with a missing-file error, and nothing tells the
caller this can happen. #2470 asks for a docs note and a runtime
warning.

Fixes #2470

## Scope

- `AsyncTable.optimize` emits a `UserWarning` when `cleanup_older_than`
rounds to 0 ms or less. `LanceTable.optimize` runs through it, so sync
callers get the warning too.
- The `cleanup_older_than` docstring on `Table.optimize`,
`LanceTable.optimize`, and `AsyncTable.optimize` gets a `.. warning::`
block, in the same style as the existing `delete_unverified` one.
- Removed the function-local `import warnings` in `AsyncTable.optimize`.
`warnings` is already imported at module level, and the local import
would shadow it for the new check (`UnboundLocalError`).
- `test_optimize` and `test_optimize_delete_unverified` intentionally
use a 0 cleanup, so they now assert the warning with `pytest.warns`.

Nodejs `cleanupOlderThan` is out of scope, since the issue is filed
against the Python SDK.

## Tradeoffs

Like the existing `retrain` deprecation warning, the warning is raised
inside the async coroutine. For sync callers it is attributed to
`asyncio/events.py` rather than the caller's line. Warning from both the
sync and async entry points would duplicate the check for a small gain.

## Verification

- Repro: a second handle checked out at an older version, then
`optimize(cleanup_older_than=timedelta(0))` on the first. The second
handle's read fails with `LanceError(IO) ... Not found:
.../data/<file>.lance`. Before this change no warning is emitted. After
it, the `UserWarning` fires.
- New tests `test_optimize_warns_on_zero_cleanup` (async, and checks
that `timedelta(days=1)` and the default don't warn) and
`test_optimize_warns_on_zero_cleanup_sync` fail on main (`DID NOT WARN`)
and pass with the fix. The failing tests are in their own commit before
the fix.
- `pytest python/tests/test_table.py -m "not slow and not s3_test"`: all
pass except `test_merge_insert`. That test's `timeout=timedelta(0)`
assertion fails the same way on unmodified main on my machine.
- `ruff format --check` and `ruff check` are clean on the changed files.

Tests ran against the source tree with the `_lancedb` extension from the
matching `0.40.0b11` wheel. The change touches no Rust.

This PR was written by an AI coding agent (Claude Code) and reviewed
before submission.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Jay Zhou <190699450+jayzhou2309@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-28 20:11:47 -05:00
brekenandbreken-ai d8063545d9 fix(node): write NaN and Infinity as SQL literals in table.update (#4327) 
## What's wrong

`toSQL` renders numbers with `toString()`, so `NaN` becomes `NaN` and
`Infinity` becomes `Infinity`. SQL reads those as column names:

```ts
await table.update({ values: { price: NaN }, where: "id = 1" });
// Error: Invalid input, Schema error: No field named "NaN". Valid fields are id, price.
```

`Infinity` and `-Infinity` fail the same way. The only workaround today
is `valuesSql`.

## Fix

Emit `CAST('NaN' AS DOUBLE)`, `CAST('Infinity' AS DOUBLE)` and
`CAST('-Infinity' AS DOUBLE)` for these values. Finite numbers render as
before.

## Tests

- `__test__/util.test.ts` adds `toSQL` cases for `NaN`, `Infinity` and
`-Infinity`.
- `__test__/table.test.ts` adds "should let me update float values to
NaN and Infinity", which updates a float column and reads the values
back.
- On `main`, the new cases fail with `No field named "NaN"`.
- With the fix, all of `util.test.ts` and `table.test.ts` pass (344
tests). These were run against the published
`@lancedb/lancedb@0.40.0-beta.11` native binary.
- `biome check` is clean on the changed files.

The Python SDK has the same issue in `value_to_sql`. That fix is in
#4326.

The fix was found and written by an AI agent (Breken) and checked
against the tests above.

Co-authored-by: breken-ai <312387581+breken-ai@users.noreply.github.com>
 2026-09-28 20:11:39 -05:00
Yang CenYang CenLance Release
 		0327618a62 feat(sql): add native vector dedup and partitioned mv refresh (#4303) 
## Summary

Materialize a source-preserving dedup result from an indexed, pinned
snapshot with a short declaration. The default greedy policy keeps the
smallest unassigned snapshot row ID and removes only its directly
matching neighbors: For A < B < C, an A–B–C chain without an A–C edge
keeps A and C.

```python
from lancedb import vector_dedup

view = db.create_materialized_view(
    "images_clean",
    vector_dedup("images", column="phash", distance_threshold=4),
)
cleaned = view.table
```

The Python source expression captures the source version once at
creation and
waits for the materialized result by default. Use `dataset_version=...`
for an
explicit snapshot, `with_no_data=True` for declaration only, or
`create_materialized_view_async(...)` for a job handle. AsyncConnection
supports
the same expression. It uses the existing MV REST endpoints and registry
jobs,
without opening a SQL query connection. Source names are exact
root-namespace
table names. Additional select/where/limit options are rejected for this
source.

The SQL declaration remains available:

```sql
CREATE MATERIALIZED VIEW images_clean AS
SELECT * FROM vector_dedup('images', 7, 'phash', 4);
REFRESH MATERIALIZED VIEW images_clean;
```

This is the host-facing SQL entry point. The MV builder now accepts a
typed native vector source; the Rust SDK also supports the same
definition through `MaterializedViewDefinition::from_sql` and
`prepare_definition`; `view.refresh().execute()` executes it locally.
Hosts distribute its units using their existing job infrastructure. The
raw `vector_duplicate_pairs` source remains available separately when
callers want the pair relation.

## Implementation and contract

- Use actual Rust Lance `13.0.0-beta.15` (`db53dcf367`); the native
partition primitive computes pairs from index representations without
source-vector aggregation or a Python pair loop.
- Extend partitioned MV refresh with two dependent waves. Physical
segment/partition tasks stream pairs through a bounded external sort,
select direct representatives and write per-source-fragment bitmap
artifacts. Source-fragment tasks start after every selection receipt
completes and stream retained original rows, including blob payloads,
into uncommitted output fragments.
- Reuse normal blob write coercion and row-mask scans, including stable
row-ID translation after compaction. Null vectors, isolated rows and
source deletions are preserved correctly. Conflicting representative
decisions across overlapping index scopes fail before publication.
- Expose `dependency_units()` and `write_refresh_partition_with_inputs`
alongside the existing plan/commit API. Receipts bind
snapshot/configuration/policy/destination generation. Only a complete
accepted set can publish; unfinished units restart, completed units can
be reused. There is no pair-position checkpoint.
- Use MV format 4 for dedup results and format 3 for native pairs;
Python and TypeScript metadata readers recognize both. No source
deletion or source row-ID migration occurs.

Only same-segment/same-partition edges are covered. Quantized scores use
symmetric represented-vector distances. Result materialization requires
V2 data storage and is initially a native MV source, not an ordinary
SELECT table function. Compressed vertex masks stay in memory; edge
sorting spills to local disk. Unreferenced artifacts use normal Lance
retention/cleanup and must be retained for the job's recovery window.

## Validation

AWS Linux validation:

- Native pair/result integration matrix passed: multiple
segments/partitions, exact thresholds, empty output, cross-batch data,
single-process pair parity, source preservation, lost receipts and
publication guards.
- Blob/null-vector/pinned-deletion tests passed with both row-ID modes,
including compaction that changes physical row addresses.
- Representative policy, forced external-sort spill and
conflicting/corrupt artifact tests passed.
- Full MV regression suite: 153 passed, 2 pre-existing ignored. Rust
formatting and Clippy passed.

- Python MV suite: 47 passed; TypeScript MV suite: 11 passed. Native
bindings were rebuilt in project-local environments; formatting, lint
and TypeScript docs passed.
- Rust declaration doctests: 3 passed, including the complete local
dedup example.
- Companion host validation: 41 MV job tests passed (1 pre-existing
ignored), full workspace Clippy passed, and the complete process-level
CLI E2E suite passed (3 tests). Recovery exercises both selection and
materialization waves.
- A real CIFAR-10 pHash run used five actual registry-worker owners,
completed 32 selection + 24 materialization tasks, and retained 49,846
of 50,000 rows with identical original values and image payloads.

Host workflow performance measurements are recorded in the companion
host PR. Cross-platform CI is still running. The previous remote-test
bootstrap failed before Cargo tests with `401 Bad credentials`; the
workflow fix described below is being validated by CI. No deployment or
merge is performed.


## Python source validation

AWS tests cover automatic and explicit snapshots, physical/stable row
IDs, local
and remote creation, sync/job/async-connection APIs, delayed refresh,
strict
options and identifier quoting. The native Python extension was rebuilt
through
the project's uv/maturin backend. Ruff, Pyright and the full MkDocs
API-reference
build passed; both new public symbols are rendered.

A real 50,000-image FEv2 run disabled both Python SQL entry points and
supplied an
unreachable SQL endpoint. The new API still materialized the same 49,846
retained
rows and original image payloads. Three recorded creation/refresh jobs
completed
56 tasks each on all five registry-worker owners, and remained on source
snapshot
3 after source latest advanced to 4. No worker or server wire-protocol
changes
were needed for this client entry point.

Previous CI authentication failure:
https://github.com/lancedb/lancedb/actions/runs/36334679276/job/108663151265.

## Review and CI fixes

- Validate all JSON merge source fields before projection, rejecting
unknown and duplicate names at every struct level. Missing/reordered
fields and ordinary JSON merges remain supported. A checked-in 12-case
integration matrix covers root fields, structs, lists, large lists,
fixed-size lists and map values; rejected writes leave the table version
and row count unchanged.
- Match namespace connection MV method signatures to
`MaterializedViewSource` in all four sync/async methods, fixing the
Python 3.10 import failure from runtime override checks.
- Restore the remote-test script's encrypted-CLI path: use the Actions
token for artifact download and pass `SOPHON_READ_TOKEN` under the
variable the decryption script consumes. The previous workflow omitted
that variable and attempted the private release path, failing with HTTP
401. On the final head, remote CI passes: 1,464 Rust unit tests, the
integration targets and doctests pass (3 unit tests ignored).

AWS validation for these fixes: both review reproducers failed before
their fixes and pass afterward; 55 merge tests, 29 cast tests and the
standalone 7-case admission regression pass. Python 3.10.21 imports
successfully; 90 MV/namespace tests, 14 JSON tests and four real
namespace dedup cases pass. After `794b97a0`, the native binding was
rebuilt again and the JSON suite rerun. Rust formatting/Clippy, Ruff and
Pyright pass. Sophon's locked test compilation passes against the final
SDK source for all seven affected crates. Extended runners, logs and
benchmark artifacts remain local.

Gatekeeper recommends approval on `794b97a0`, and the `breaking-change`
label is set. Final CI: **49 checks passed, 3 expected skips, no
failures or pending checks**, including remote tests, Python 3.10,
minimum dependencies and Windows ARM Rust tests. All wheel builds pass
after retrying the Linux matrix affected by transient AlmaLinux
repository-metadata 404s. All code-review threads are resolved. The PR
remains open for maintainer review and merge.

---------

Co-authored-by: Yang Cen <yangcen@Yangs-Mac-mini.local>
Co-authored-by: Lance Release <lance-dev@lancedb.com>
 2026-09-28 18:06:55 +08:00
Ayush Chaurasia 139382b5f7 feat(python): pass batch_size and other CrossEncoder kwargs (#4328) 2026-09-25 21:50:55 +05:30
LanceDB Robot d0fb43b0b4 chore: update lance dependency to v13.0.0-beta.14 (#4324) 
Update the Rust workspace Lance dependencies and Java lance-core from
v13.0.0-beta.12 to
[v13.0.0-beta.14](https://github.com/lance-format/lance/releases/tag/v13.0.0-beta.14),
including the refreshed Cargo lockfile.

No compatibility fixes were required; `cargo clippy --quiet --workspace
--tests --all-features -- -D warnings`, `cargo fmt --all --quiet`, and
`git diff --check` passed.
 2026-09-25 09:02:00 -05:00
Dan RammerandClaude Opus 5 2216586023 feat(query): dedup hybrid search by primary key on MemWAL tables (#4268) 
Implementing a PK-based dedup to support WAL in hybrid search.
Basically, the existing code just calls a vector query + FTS query and
dedups locally using the row ID. Since row ID is unsupported in WAL
(currently), we update the algorithm to use the tables PK (which is
required in WAL, and uniquely linked to a PK on updates). To enable
PK-dedup branch, we've chosen to fallback on error rather than fire an
exploratory query. Specifically, we issue the regular hybrid query,
using row IDs, and when that fails with "WAL does not support row IDs"
we fallback to the PK-based variant, and then cache the fallback for
future calls from the same client. The alternative is that we chec if
WAL is enabled before issuing the query. This would require a
`get_lsm_write_spec` call, slowing down all the existing non-WAL paths
unnecessarily.

The ultimate goal is to get row ID supported in WAL, but that is pending
stable row ID GA and subsequent integration into the enterprise stack.
Once that lands, we will rip out the PK-based dedup that is only for
WAL-enabled tables and use the existing row ID code. This is the main
reason for building this adaptively. I had spec-ed out a refactor so
that rather than duplicating code between python and rust we build this
in rust and have python call that through PyO3. However, that made the
reverse path (ie. ) more difficult. So I think the tradeoff of
maintaining duplicate code in the short term for the ease of reverting
this in the medium term (hopefully 2-3 months).

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-24 22:02:04 -05:00
Wyatt Alt 604b1b583b feat: pause and resume jobs from the Python SDK (#4112) 
Cancellation is the only job control the SDK exposes, and it is
terminal, so a long-running server-side job cannot be parked and picked
up again. This adds `pause_job` and `resume_job` to the Rust core
connection and the Python bindings (sync and async), posting to the
server's `/v1/jobs/pause` and `/v1/jobs/resume` endpoints.

A pause parks the job until it is resumed: its workers drain and stop.
The outcome strings mirror the server's answers — a job finalizing its
results reports `"committing"` and cannot be parked, and a resume before
the drain is confirmed reports `"still_pausing"`; both are retried
rather than failed. Resuming re-queues the job and its workers pick
their work back up from checkpoints. Local connections report the
operations as unsupported, like the rest of the jobs API.
 2026-09-24 23:47:00 +08:00
Lance Release 11c637252f Bump version: 0.40.0-beta.10 → 0.40.0-beta.11 2026-09-24 10:05:13 +00:00
Xuanwo 0b18dcdaec fix(python): warn instead of failing on local imports in @udf source (#4318) 
Since 0.40.0b8 (#4254), `@udf` raises when the packaged source imports a
module from a local source tree that is neither shipped with `code=` nor
named by a declared pip/conda package. That turned registrations that
used to succeed into errors:

- Callers that rewrite the packaged source before registering it now
fail at decoration. Sophon's datagen integration suite inlines its
test-module helpers this way, and
`test_registered_udf_source_stands_alone_on_a_worker` fails on 0.40.0b8.
- The check cannot know which modules a distribution installs. A
declared package whose distribution name differs from its module (for
example an internal library installed editable) is reported as local,
and there was no way to proceed.

This keeps the diagnosis but makes it a `UserWarning`, so registration
behaves as it did before #4254 and still tells the author, at
registration time, why the worker is likely to fail.
 2026-09-24 18:02:25 +08:00
Taylor a6dcbba49e feat(python): add opt-in request batching to TypeSafeReranker (#4316) 
Adds opt-in `batch_size=40`: 80 non-null candidates use **2 requests
instead of 80**. Default `batch_size=1` preserves the existing payload.
Each batched question sees only its document and the shared query;
instructions/criteria stay unchanged. Prompts referencing
`state.document` need adaptation. Concurrency still limits requests;
retries remain SDK-managed. Batch-local IDs and ordered results preserve
duplicate documents; invalid IDs/probabilities raise without fallback.

Validation: **124 automated tests passed**, plus live search
integrations in both modes. A **100-query live subset test of the
initial implementation** (6,851 candidate pairs; `jev-1.13.0`, SDK
0.7.1, concurrency 32) passed: unbatched/batched calls **6,851/199**,
median **877/351 ms**, p95 **9,446/598 ms**. Hybrid Hit@5/Hit@10 was
**83%/89% vs 84%/90%**; vector/FTS results and exact configuration are
in the [test
report](https://github.com/lancedb/lancedb/blob/typesafe-request-batching/python/benchmarks/typesafe_batching.md).
Ruff and MkDocs passed. The simplified batching flow also passed a fresh
four-query live check: 292 candidate pairs, with 292 unbatched versus 8
batched calls.

Builds on #4209; motivated by [research
#7](https://github.com/lancedb/research/pull/7).
 2026-09-24 13:53:52 +05:30
Lance Release b3f0e12e32 Bump version: 0.40.0-beta.9 → 0.40.0-beta.10 2026-09-24 07:50:23 +00:00
Jack Ye e4d0f9e2da feat: return a cleanup job from drop_view (#4262) 
Dropping a view unbinds its name and leaves the definition dataset to a
server-side cleanup job, so the two are separate events a caller may
want to wait on.

`drop_view_async` returns that job — the same shape
`drop_materialized_view_async` and `drop_function_async` already use: a
`202` carries the job id, a `200` (nothing was bound to the name) yields
an already-finished job with no id, and any other success status is an
error rather than a silent no-op.

## `drop_view` waits

`drop_view` now awaits the job before returning, so a caller who does
not want to think about cleanup gets the stronger guarantee: when it
returns, the definition really is deleted.

That is deliberately **different** from `drop_materialized_view` and
`drop_function`, which return as soon as the name is unbound and
document that content may still be deleting. The view API is the newer
one, and waiting is the semantic worth having; the other two are left
alone rather than changing behaviour already released.

## Surfaces

`Database` trait, the remote client, `Connection`, and the Python and
Node bindings — matching where `drop_materialized_view_async` is already
exposed.

Four client tests cover the accepted case reporting its job id, the
nothing-bound case reporting a finished job, a `202` without a usable
`job_id`, and an unexpected success status.
 2026-09-24 00:27:20 -07:00
Lance Release 85c9cf6240 Bump version: 0.40.0-beta.8 → 0.40.0-beta.9 2026-09-23 23:40:53 +00:00
LanceDB RobotLu QiuClaude Fable 5.1
 		53a2afdffa chore: update lance dependency to v13.0.0-beta.12 (#4265) 
Update the Rust workspace Lance dependencies and Java lance-core from
v13.0.0-beta.8 to
[v13.0.0-beta.12](https://github.com/lance-format/lance/releases/tag/v13.0.0-beta.12),
refreshing Cargo.lock.
No compatibility fixes were required; validation passed with `cargo
clippy --quiet --workspace --tests --all-features -- -D warnings`,
`cargo fmt --all --quiet`, and `git diff --check`.

---------

Co-authored-by: Lu Qiu <luqiujob@gmail.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-23 14:31:35 -07:00
lancedb-gatefixer[bot] 85daeb27cb fix: isolate remote table checkout state per handle (#4310) 
## Cause

`RemoteDatabase` cached and returned the same `Arc<RemoteTable>` for
every open of a table on one connection. Checkout mutates version and
freshness state on that shared object, so pinning one handle also pinned
the others.

## Fix

Cache the table's server version rather than its mutable handle. Each
`open_table` now creates its own `RemoteTable` with independent
checkout, schema cache, and freshness state.

## Validation

- Added a cache-enabled regression test that failed before the fix:
pinning one handle changed another handle's reported version. It now
verifies a write through the independent handle succeeds.
- `cargo test --quiet --features remote -p lancedb --lib
remote::db::tests` (90 passed)
- `cargo check --quiet --features remote --tests --examples`
- `cargo clippy --quiet --features remote --tests --examples`
- `cargo fmt --all`

Fixes #4296

<!-- lance-gatekeeper-fix:v1 agent=38593bfbce0279286da9850fe61050e0
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-09-23 14:27:44 -07:00
lancedb-gatefixer[bot] 75e71176cb fix(python): prevent BlobFile finalizer panics on Tokio workers (#4291) 
## Cause

`io.RawIOBase.__del__` checks `closed` and calls `close()` during GC.
Both PyO3 methods entered `block_on`, which panics when GC runs on a
LanceDB Tokio worker.

## Fix

Track Python blob handle closure atomically so `closed` is synchronous.
`close()` marks the handle closed immediately and, when called on a
Tokio thread, schedules the underlying async cleanup without blocking
that thread. Reads and seeks after close fail immediately; ordinary
callers still wait for cleanup.

## Validation

- Rebuilt the Python extension with `uv run --extra tests --extra dev
maturin develop --extras tests,dev`.
- Ran the blob test file and remote blob handle test: 69 passed, 1
skipped.
- Ran `cargo fmt --all`, `ruff format .`, and `ruff check .`.

Fixes #4283

<!-- lance-gatekeeper-fix:v1 agent=15a7bd545e63dea4b69faa0429f0a684
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-09-23 14:24:04 -07:00
lancedb-gatefixer[bot] 1842c88f2c fix(node): reject unsupported object row values (#4307) 
## Cause

Schema inference treated unrecognized objects as nested structs. Objects
without enumerable fields produced no field paths, so schema matching
dropped their columns before Arrow or blob conversion. Blob instances
with own fields were mistaken for struct fields.

## Fix

Walk only nonempty plain records as structs. Reject unsupported object
values in scalar fields, nested structs, and list elements with the
field path and row number. Preserve the object types already passed
through to Arrow. Support for converting these new input types is
tracked in #4270.

Nested class instances with enumerable fields previously became structs;
this change rejects them and is labeled `breaking-change`.

## Validation

- `pnpm build`
- `pnpm lint`
- `pnpm test __test__/arrow.test.ts __test__/blob.test.ts
__test__/table.test.ts --runInBand --silent` (626 passed)
- `pnpm run docs`

Fixes #4269

<!-- lance-gatekeeper-fix:v1 agent=426011291c209da027689ae88a0b4da7
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-09-23 14:22:40 -07:00
Lance Release df7bfcdb65 Bump version: 0.40.0-beta.7 → 0.40.0-beta.8 2026-09-23 17:05:35 +00:00
Xuanwo a9e350c158 ci(python): build Windows release wheel on 8-core runner (#4264) 
Since #4241 restored fat LTO / 1 codegen unit for the Windows wheel, the
`windows` job in PyPI Publish has been cancelled at its 90-minute
timeout on every run, including the v0.40.0-beta.6 and v0.40.0-beta.7
tags, so neither release was published. In those runs dependency
compilation finishes after ~26 minutes and the `lancedb` crate compile +
fat-LTO step was still running after 63 minutes.

Move the job to the org's `windows-2025-8x-x64` larger runner and raise
the timeout to 150 minutes. Fat LTO stays, since the thin-LTO wheel
exceeds PyPI's 100 MiB limit.

The final fat-LTO step is largely single-threaded, so the gain from 8
cores is mainly in the parallel dependency phase plus faster/larger
hardware; the PR run of this workflow is the first measurement of fat
LTO on this runner.
 2026-09-24 01:04:18 +08:00
Xuanwo b944055b63 feat(python): class Functions with initialization and shipped helper code (#4254) 
Stacked on #4253.

Remote Python Functions could not share helper code, take typed
parameters, or keep per-process state: helpers became `from <module>
import ...` lines a worker cannot resolve, every parameter was an `env=`
string, and state had to be stashed on imported modules. MMLB's OpenAI
preset shows the cost — 24 env vars, a 302-line callable, and a rate
limiter duplicated between the scalar and batch variants, one of which
never constructed it (ENT-2516).

**Class Functions.** `@udf` accepts a class. Each remote instance runs
`__init__` once, calls `__call__` for every row or batch it processes
(row or batch mode is inferred from annotations as before), and calls
`close()` once if defined. Calling the definition locally constructs the
class, so unit tests stay ordinary.

**Initialization.** The annotated `__init__` parameters become the
Function's initialization fields. A binding passes their values beside
its column inputs: `fn(text=col("body"), model="small",
dimensions=512)`. Values are constants of that binding; another column
can bind the same Function version with other values. Types are limited
to booleans, integers, floats, strings, and lists or structs of them,
which have one unambiguous JSON and SQL spelling. A parameter with a
default may be omitted; a null value then takes the default.
Initialization and input names must be disjoint because both are keyword
arguments of one call. Secrets stay on `EnvVarSecret`.

**Helper code.** `code=[package, ...]` ships top-level modules or
packages as Python source in a `python_bundle` artifact (a canonical
JSON object of path → source), imported normally on the worker. Changing
a helper changes the artifact digest and so the Function version; the
environment from `pip`/`conda` is reused. Functions and classes defined
in `__main__` (notebook cells, scripts) are packaged by source,
recursively and dependencies first. An import of a module that lives in
a local source tree, is not shipped with `code=`, and names no declared
package is now rejected at registration instead of failing on the
worker. Closures, lambdas, and nested definitions are still rejected,
with the reason.

A Function without these features packages byte-identically to before
(`python_callable`, same digest, no `initialization` on the wire). Class
sources are located through a method's code object, because
`inspect.getsource` cannot find a class defined in a notebook cell or
doctest.

The packaging contract is documented on `udf`. Sophon changes that build
and execute these artifacts are in lancedb/sophon; they were verified
end to end on a Linux local server (registration → REST, SQL, and
materialized-view bindings with different initialization → refresh →
query, one construction per instance).
 2026-09-24 00:56:52 +08:00
Wyatt Alt 5538bdb9ff feat: refresh an ivf-grouped materialized view in units (#4232) 
A grouped view's refresh runs its whole aggregate in one process, so a
view grouped by ivf_partition(col) over a large table is bounded by a
single worker however many workers a deployment has. The IVF index
already holds each partition's row ids, so the aggregate splits along
the index without a shuffle: every partition's groups are computed from
its own rows alone.

plan_grouped_refresh names the units (one per index partition plus one
for the rows the index cannot place) and the source version they read;
write_grouped_unit computes one unit's groups from the index's row ids,
plus the rows of fragments the index has not covered, assigned in place,
and writes them as uncommitted fragments; commit_grouped_refresh
replaces the view's rows with every unit's fragments in one Update.

The split has to publish what the single-pass refresh would. A unit
applies the view's predicate in its own query, because neither the
indexed take nor the fragment scan filters the way a lance scan does. An
index that does not say which fragments it covers has unknown coverage,
not empty, so it yields no plan at all and the caller refreshes in one
pass rather than reading those rows twice. Each result names its unit,
its plan and the view incarnation it was computed for -- two views of
one shape reach the same counters, and fragments written into one
dataset are not publishable into another -- and the commit publishes the
plan's units exactly once each or nothing. The commit lands on the
planned generation or is refused: lance rebases this Update over a
concurrent append rather than rejecting it, so the version it actually
landed on is checked, as the single-pass rebuild already does.

What holds every unit to one index is the source version the plan pins:
indices live in the source manifest, so a rebuild lands in a version the
units never read. Within that version a segment's postings can still
outlive its ownership -- a column rewrite attaches a new file and takes
the fragment out of the segment's bitmap without dropping its rows from
the posting lists -- so a unit keeps a segment's rows only while it
holds their fragment, and reads the rest from the scan.

The split is the grouping only where the index assigns by its own
centroids. lance also builds an index from precomputed partitions, and
records nowhere that it did, so a posting list can hold a row that
ivf_partition puts elsewhere -- that row's group would then be
aggregated in its own unit as well and published twice, since
concatenated fragments cannot merge two halves of a group. The plan
samples each partition and yields no units when they disagree, every
unit proves the rows it took before grouping them, and the commit
refuses a unit that wrote more than the single group its key allows.
 2026-09-23 09:13:17 -07:00
Xuanwo 3728d41f02 feat: carry Function initialization in the canonical model (#4253) 
A Function instance is created once with an initialization row
(`create(initialization, context)` in the Function Format), but nothing
in the client model could say what that row holds or where its values
come from, so every source-built Function ran with an empty row and
users pushed configuration through `env=` strings instead (ENT-2516).

This adds initialization to the canonical wire model without changing a
Function's identity:

- `FunctionSignature.initialization` lists the fields of the row, in
order. It is omitted when empty, so existing signatures and
FunctionVersion hashes are unchanged.
- `FunctionApplication.initialization` carries a binding's values as
JSON. The service validates them against the Function's fields and
re-encodes them as Arrow, so this JSON is transport only and never
hashed; that is why floats are accepted here while column-input literals
keep the Slice 1 domain.
- `FunctionBinding.initialization` is the validated one-row Arrow IPC
stream (base64) that every instance of the binding is created with.
Values belong to the binding, so one Function version can back several
columns with different values.
- The binding-shape check that guards schema mutations accepts the new
field; without it a table holding an initialized binding refused further
declarations.

Rust and Python share new canonical goldens for an initialized
application and binding.

The authoring side (class `@udf`, initialization arguments, shipped
helper code) is the stacked PR on top of this one.
 2026-09-23 23:31:49 +08:00
Lance Release f359c5c8c5 Bump version: 0.40.0-beta.6 → 0.40.0-beta.7 2026-09-23 07:28:56 +00:00
Jack Ye 8541d6d9df feat: add view CRUD APIs (#4236) 
A view is a named query a database stores and plans on every read. It
holds no rows, which is the whole difference from a materialized view.

## API

| Verb | Route |
| --- | --- |
| `create_view(name, query, namespace_path)` | `POST
/v1/view/{id}/create` |
| `describe_view(name, namespace_path)` | `POST /v1/view/{id}/describe`
|
| `drop_view(name, namespace_path)` | `POST /v1/view/{id}/drop` |
| `list_views(namespace_path)` | `GET /v1/namespace/{id}/view/list` |

On `Connection` and the `Database` trait, with the remote client, Python
(sync and async) and Node bindings. Local databases return
`NotSupported`: the server side is Sophon's, where a view is an object
of the database manifest.

`ViewDescription` carries the defining query, the database *and
namespace path* unqualified names in it resolve against, and the schema
the query resolved to. `create_view` returns one, so a caller has the
schema without a second call.

Both defaults travel with the view because it outlives the session that
declared it: the server re-plans the stored query on every read, so a
reader resolving an unqualified name against its own defaults would read
a different table. `default_namespace_path` crosses the wire as
`default_namespace`, a path like `namespace`, absent for the root.

There is no replace: a name already taken is an error, and changing a
view is a drop followed by a create, each authorized against what it
actually touches.

Querying a view stays SQL's job. There are no rows behind a view, so
there is no `open_view` returning a `Table`.
 2026-09-23 15:09:40 +08:00
Rudra Prasad Bhuyan dd2539c2ca fix(python): convert objects for JSON fields (#4211) 
## Summary

Closes: #4060 

  - Convert Python dict/list objects to JSON strings during ingestion.
  - Support JSON fields nested inside structs and  lists.
  - Cover `add` and `merge_insert` JSON ingestion paths.
  - Add regression coverage for nested struct and list JSON fields.

  ## Testing

  - `git diff --check`
  - Python compilation passed.
- Focused pytest was attempted but could not complete because the native
extension build stalled during `uv` bootstrap.
 2026-09-22 14:06:11 -07:00
Lance Release a6a6617ee2 Bump version: 0.40.0-beta.5 → 0.40.0-beta.6 2026-09-22 18:00:27 +00:00
Wyatt Alt e020e13744 feat: group a materialized view by the IVF partition of a vector column (#4223) 
Grouping near neighbours together, so an all-pairs comparison runs per
bucket instead of over the whole table, needs the partition an IVF index
assigns each vector.

`ivf_partition(column)` returns that partition, assigned by the
centroids and distance type of the IVF index on the column. It is bound
when the view is declared and again at every refresh; an index retrain
commits a new source version, so the next refresh regroups. A column
without an IVF index, or with two, is refused.


Stacked on #4222.
 2026-09-22 10:16:20 -07:00
Jack Ye 814de30c5a feat: return a cleanup job from drop_function (#4237) 
Dropping a Function can leave its content to a server-side cleanup job,
so the name drop and the content deletion become separate events a
caller may want to wait on.

`drop_function_async` returns the unbind result alongside a `Job` for
the cleanup, the same shape `drop_table_async` and
`drop_materialized_view_async` already use: a `202` carries the job id,
and a `200` — an inline deletion, or a name that was not bound — yields
an already-finished job with no id. A `202` without a usable job id is
rejected rather than silently reported as finished.

`drop_function` keeps its `bool` result and now delegates, so nothing
changes for callers that do not care when the content goes.

Available on `Connection` in Rust and on both the sync and asyncio
Python connections.
 2026-09-22 09:54:23 -07:00
Wyatt Alt 254688df96 feat: group a materialized view by GROUP BY (#4222) 
A view could only map source rows one to one, or one to many through a
FROM-position item, so nothing could see all the rows sharing a key.

A view query now takes `GROUP BY expr, ...` with aggregate projections,
planned and executed by DataFusion over the lance scan. A group spans
fragments, so a grouped view is recomputed in full whenever its source
changes; each row's provenance is its group's smallest source row id.
Such a query is stored as definition format 2, so a reader that predates
grouping reports the view as unrefreshable instead of failing to parse
it.


Stacked on #4190.
 2026-09-22 08:03:27 -07:00
Xuanwo 518d7ff1dd fix: keep REST server adapter out of production dependencies (#4240) 
The production `remote` client talks to an existing namespace over HTTP.
It needs `lance-namespace-impls/rest`, not the REST *server* adapter.
`RestAdapter` is only used by `cfg(test)` integration tests that stand
up an in-process server. Enabling `rest-adapter` on the production
`remote` feature pulled that server stack, including Axum 0.7, into
default Python wheels.

This keeps `rest` on `remote` and moves `rest-adapter` to a
`lance-namespace-impls` dev-dependency so those tests still compile and
run. Cargo resolver=2 does not leak the extra feature into
`lancedb-python`.

## Measurement

Paired `maturin build --release --strip --target aarch64-apple-darwin
--features fp16kernels` wheels. Source was `3be29228` plus this
Cargo.toml change, which is this PR's tree (`878b2ae5` on `3be29228`).
Same toolchain, profile, and packaging flags; only `rest-adapter` moved.

| Artifact | Before | After | Delta |
| --- | ---: | ---: | ---: |
| Compressed wheel | 64,685,664 | 63,824,281 | −861,383 (−1.33%) |
| `_lancedb.abi3.so` uncompressed | 148,255,440 | 146,275,936 |
−1,979,504 (−1.34%) |

This is macOS arm64, not Windows. It does not resolve the Windows wheel
upload limit. Tonic still pulls Axum 0.8; the change only removes the
adapter's Axum 0.7 stack from the production graph.
 2026-09-22 15:13:51 +08:00