rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-10-30 15:26:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,336 Commits 364 Branches 568 Tags
84fc48198b918b7a399475fc53eb011efae14087
Commit Graph
3336 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nishil Rathod 84fc48198b docs: add guide for using LanceDB in VS Code extensions 
Closes #1849

This was lancedb/docs#371. After lancedb/docs#355 moved the open-source pages, their navigation and the snippet tests into this repository's `docs/web` and `docs/web-tests`, that PR could no longer merge there, so I've moved the same change here.

## Why

#1849 asks for an example of LanceDB in a VS Code extension, because people kept running into problems there (#927, #1713). An earlier attempt, #3682, added a README link. @prrao87 asked for something else instead: a quick guide on the docs site, under **Integrations › AI Platforms & Frameworks**, showing how to use LanceDB in an extension and what results look like. The author of #3682 agreed and closed it. This PR is that guide.

## What it adds

- **`docs/web/integrations/ai/vscode.mdx`**: a new page, also added to `docs/web/docs.json`. It builds a small extension with two commands. One indexes the workspace's Markdown into a LanceDB table using the built-in `huggingface` embedding function, so no API key is needed. The other searches it by meaning. The page covers the parts that are specific to VS Code:
  - storing the table under `context.storageUri` instead of a relative path, which is the cause of #927;
  - `@types/node` ≥ 22 and `skipLibCheck`, which covers #1713 and the DOM-type errors from `apache-arrow`/onnxruntime typings;
  - pinning `apache-arrow` inside LanceDB's peer range;
  - marking `@lancedb/lancedb` and `@huggingface/transformers` as esbuild externals, and removing `node_modules/**` from the template's `.vscodeignore`;
  - storing each file's URI so a hit opens in the right folder of a multi-root workspace;
  - building one VSIX per platform with `vsce package --target`, with a table mapping each target to LanceDB's native package, and a note that the sample's local model (via `onnxruntime-node`) doesn't run on Alpine;
  - keeping the Transformers.js model cache in `globalStorageUri`, which only works if it's loaded with `import()`;
  - a troubleshooting table listing each error message with its cause and fix.
- **`docs/web-tests/ts/vscode_search.ts`** and **`vscode_search.test.ts`**: the LanceDB code the page shows (connect, chunk, index, search), plus jest tests for it.
- **`docs/web-tests/ts/vscode_extension.ts`**: the extension's `src/extension.ts`. Jest can't run it because it needs the VS Code extension host, so it was tested in a real extension instead (see below). This follows the precedent of `integrations.ts`.
- **`docs/web/snippets/vscode_search.mdx`** and **`vscode_extension.mdx`**: generated with the command in CONTRIBUTING.md.

## Changes from lancedb/docs#371

- **New paths only.** The page, tests and snippets are the same content, at their paths in this repository.
- **One link.** `/tables/update` doesn't exist under `docs/web`, so the `mergeInsert` link now points to `/ingestion/merge-insert`.
- **Heading anchors.** Every heading now has an explicit `{#anchor}`, as CONTRIBUTING.md asks.

## How I tested it

- **Snippet tests.** In this repository at `c2c8269f`, `vscode_search.test.ts` passes (2 tests) with the pinned `@lancedb/lancedb` 0.31.0. On Windows the `npm test` script fails before it starts, because it runs `node` on the `.bin/jest` shell shim. I ran `node --experimental-vm-modules node_modules/jest/bin/jest.js --testEnvironment jest-environment-node-single-context vscode_search` instead.
- **Lint.** `biome ci` is clean for the three new files, and they carry the SPDX header that `license-header-check.yml` looks for.
- **Snippets.** I regenerated them with CONTRIBUTING.md's command, `mdx_snippets_gen.py -s docs/web-tests/py -s docs/web-tests/ts -s docs/web-tests/rs -o docs/web/snippets`, run with `python` since I don't have `uv`. It reproduces the two new snippet files exactly, once the Windows CRLF endings are normalised. As CONTRIBUTING.md warns, it also rewrites `build_with_ai_agents.mdx`, `integrations.mdx`, `search.mdx` and `storage.mdx`. I left those out.
- **Links.** `mint broken-links` from `docs/web` reports no entries on the new page, and its three internal links (`/embedding/quickstart`, `/ingestion/merge-insert`, `/search/full-text-search`) resolve to pages here.
- **Real extension.** I built a sample extension from exactly the code on the page, scaffolded with `yo code` (TypeScript + esbuild) and using `@lancedb/lancedb` 0.39.0. I ran it with `@vscode/test-cli` in VS Code 1.140 on Windows x64. The integration test runs both commands against a six-file workspace and checks the top hits. The "Sample results" block on the page is copied from that run, and after the review fixes below it reproduces line for line in VS Code 1.141.
- **Multi-root workspace.** A second test configuration opens two folders that both contain `docs/auth.md`. It types a query and picks the top hit through the real picker, then checks that the right folder's file opens at the right line. It fails on the code before the review fix and passes after it.
- **Packaged VSIX.** I packaged `vsce package --target win32-x64` (200 MB, or 148 MB with the onnxruntime trimming in the Tip). Then I unpacked it outside the project and ran the same tests against the unpacked copy, so they couldn't pick up the project's `node_modules`. They pass.
- **Negative checks behind the troubleshooting rows:**
  - With `node_modules/**` left in `.vscodeignore`, activation fails with `Cannot find module '@lancedb/lancedb'` and the commands report `command '…' not found`.
  - Without the externals, esbuild fails with `No loader is configured for ".node" files`.
  - Installing LanceDB into a project that already has `apache-arrow` 21 fails with `ERESOLVE`.
  - `require("@huggingface/transformers")` returns a different `env` object from `import()`.
- **Rendering.** For lancedb/docs#371, `mint dev` rendered the page, its 14 code blocks and the sidebar entry. I haven't re-rendered it from `docs/web`; only the paths, one link and the heading anchors changed.

**Not tested:**
- macOS. I couldn't reproduce the read-only working directory from #927; the page's fix (an absolute storage path) doesn't depend on the OS.
- Linux and macOS VSIX builds.

## Notes for reviewers

- I added `_distance` to `.select(...)`, because LanceDB warns that it will stop adding the column automatically.

## AI disclosure

I used Claude Code to research the issue and its history, write the sample extension, the tests and this page, move the change here, and draft this description. I ran every check listed above myself, and the outputs quoted on the page come from those runs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
 2026-10-29 05:25:00 +08:00
Will JonesandClaude Opus 5.5 4d7a64712f docs: adopt Embedded/Distributed terminology across docs and SDKs (#4713) 
## Summary

Adopts the product terminology guide across the docs site pages, SDK
docstrings and error messages. "LanceDB OSS" and local mode become
**LanceDB Embedded**. "Remote" as the execution model becomes **LanceDB
Distributed**, which LanceDB Enterprise provides. Where free vs paid is
the point, pages contrast "LanceDB Embedded (open source)" with "LanceDB
Enterprise". This is a first trial of the Embedded name, so reviewers
can judge it in context.

- **Embedded vs Distributed** for where execution runs (`optimize()`,
consistency, branching, maintenance). Main pages define it once:
"LanceDB Enterprise runs LanceDB Distributed: …"
- **LanceDB Cloud** is kept only for legacy-Cloud specifics (`db://`
region, the brute-force KNN limit) and marked legacy. "Cloud and
Enterprise" becomes Enterprise or Distributed.
- **Plan executor** becomes "executor" or the fleet name.
- **Error messages** change, e.g. `optimize is not supported for remote
tables.` → `... in LanceDB Distributed.` Tests that match on them are
updated.
- **Vale rules** (`EmbeddedName`, `OSSProduct`, `RemoteExecution`,
`DistributedEnterprise`, `LegacyCloud`, `InternalNames`) enforce the
terms in prose.

Unchanged: identifiers (`availability.oss`, the `remote` module and
feature, `RemoteTable`, `db://`), heading anchors, internal code
comments, and Geneva.

## Evidence

- Rust lib tests for the changed messages and the Python tests in
`test_remote_db.py`, `test_materialized_views.py` and `test_table.py`
pass locally.
- Not run locally:
- The Node tests: `remote.test.ts` and `table.test.ts` have updated
assertions.
  - Vale.
  - The `experimental_functions`-marked materialized-view tests.
- `docs/src` API reference was hand-edited to match the docstrings
rather than regenerated.

## Merge Risk

**Door:** two-way; prose and message text only.
**Blast Radius:** `docs`, `error-messages`: callers matching on old
error strings would break.

Companion PRs: lancedb/docs (assembler labels, STYLE.md) and sophon
(Enterprise pages).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-28 14:04:41 -07:00
Wyatt Alt b4cbaf2f67 docs: show stateful class Functions 
The Functions page only showed a plain function, so it never said where a class's `__init__` runs. Users loading a large model need to know it is not the client: registration reads the signature and ships the source.

This adds a section on class Functions: `__init__` runs once per instance on the worker, then `__call__` per row or batch and `close()` on retirement, state is per instance, and `__init__` parameters are initialization fields set per binding. The example's test asserts that registering the class does not construct it.
 2026-10-29 04:41:23 +08:00
JiMyung Lee 057a39a1c7 docs: row counts in list_indices can be set for remote tables 
Remote list_indices fills num_indexed_rows / num_unindexed_rows from the
server's enriched response and only leaves them unset for servers that
do not report them. The Node and Rust documentation previously implied
those fields were always unavailable for remote tables.

Synchronize the generated TypeScript API reference with the Node source
comments.

Closes #4301

Co-authored-by: desty <lee.ji.myung@gmail.com>
 2026-10-29 04:03:30 +08:00
Joshua Snider edb79d38e3 docs: correct the documented default retry statuses (#3964) 
#2349 changed the default retryable status codes to `[409, 429, 500, 502, 503, 504]`, but only the Rust docstring was updated. The Python and TypeScript bindings — and the generated TypeScript API reference — still document the old `[429, 500, 502, 503]`.

The actual default is set in `ResolvedRetryConfig::try_from` in `rust/lancedb/src/remote/retry.rs`, and `rust/lancedb/src/remote/client.rs` already documents it correctly. This brings the other three in line.

Docs only — no behavior change.

Split out from #3963 per the "one problem at a time" guidance in `CONTRIBUTING.md`.
 2026-10-29 03:05:40 +08:00
J Ganesh Kumar Reddy 12d86ea251 feat(nodejs): add col() expression builder for camelCase column names 
## Summary

Fixes #3211 by exposing the shared Rust/DataFusion expression builder through Node bindings. Case-sensitive references such as `col("firstName")` can be composed and passed to `where()`, while SQL strings remain supported.

The expression-object representation follows [the design requested on #3290](https://github.com/lancedb/lancedb/pull/3290#discussion_r3632371998). That earlier PR remains open; no claim is made that the issue was unclaimed or that maintainers have selected this implementation.

## Review corrections

- Construct the named native struct correctly and parse casts through `arrow_schema::DataType`.
- Preserve numeric values: use Int64 only for JavaScript safe integers; retain Float64 otherwise. Explicit `litInt` rejects unsupported numbers with a documented `RangeError`.
- Correct test syntax/API calls and align SQL assertions with the shared dialect, including reserved and case-sensitive names.
- Use typed, isolated temporary test tables with cleanup, and cover an actual large-number filter threshold.
- Document the public methods and commit generated API documentation.

Synced with current main `0bceaaf3` through a normal merge. The diff is limited to seven Node source/test/config files and eleven generated documentation files; no Rust core change.

## Validation

[Linux validation run](https://github.com/Ganesh1729-ui/lancedb/actions/runs/38244794288) on Node 24 / Rust 1.97.0 with the repository's `ci` profile and `remote,lancedb/metrics-otel` features:

- Rebuilt native addon, TypeScript build, Biome, Rust formatting and native Clippy pass.
- Expression/query/Arrow/table suites: **1,129 passed**; expression suite repeated: **42 passed**.
- Using the original JavaScript numeric dispatch with the rebuilt addon: **8 regressions fail, 2 controls pass**.
- Strict TypeDoc generation passes. Final local typing/docs/lint checks pass; the PR source equals the validated source except for the separate validation workflow, which is excluded from this PR.

Full provider/model/S3, other feature combinations and OS/Node matrices were not run in this follow-up. Upstream checks and maintainer review remain separate.

## AI assistance

AI was used for implementation, review and testing. No prior human review is claimed.
 2026-10-29 02:18:16 +08:00
Lance Release ece54d8917 Bump version: 0.41.0-beta.16 → 0.41.0-beta.17 2026-10-28 17:31:48 +00:00
Brendan Clement 882be89778 docs: clarify how to contribute website documentation 
The contributor guide mixes up shared documentation, Enterprise placeholders and the SDK reference. It also describes section merging that the assembler doesn't do.

This replaces the overlapping instructions with guidance on where to write, how whole-page replacement works, how to preview changes and how to keep unfinished docs hidden. It explains that source changes enter a saved release edition before they appear on staging, and points SDK contributors to the existing reference guide.
 2026-10-29 00:40:13 +08:00
alexcastgonzClaude Sonnet 5.5Xuanwo
 		659ea11ac4 fix(embeddings): default Cohere to embed-multilingual-v3.0, the v2.0 models are retired (#4698) 
Cohere retired `embed-english-v2.0`, `embed-english-light-v2.0` and
`embed-multilingual-v2.0` on April 4, 2026
([deprecations](https://docs.cohere.com/docs/deprecations), which lists
`embed-english-v3.0`, `embed-multilingual-v3.0` and `embed-v4.0` as the
embedding alternatives). `CohereEmbeddingFunction` still defaulted to
`embed-multilingual-v2.0`, so every call without an explicit `name`
fails.

The default is now `embed-multilingual-v3.0`, the multilingual v3 model,
and the retired models are removed from the docstring and the Cohere
integration page (the page also listed `embed-english-v2.0` as the
default, which did not match the code). The docs example, its generated
snippet, the docs test and the slow Cohere test use the new name.

This changes the default vector size: `ndims()` returns 1024 for
`embed-multilingual-v3.0`, where the old default gave 768. Tables
created from a schema that uses `cohere.ndims()` get 1024-dimension
vectors from now on. A table that already holds 768-dimension vectors
from the retired model needs a re-embed with a current model, since the
old model can no longer be called. The `ndims()` entries for the v2
names are left as they are.

Test: `test_cohere_default_model_is_not_a_retired_v2_model` in
`python/python/tests/test_embeddings.py` checks the default name and
that `ndims()` is 1024. The repository's pytest session could not be run
for this change (its `conftest.py` needs a newer native module than the
released wheel); the body of that test was run against the released
`lancedb` 0.40.0 with this version of `cohere.py` in place, where it
passes, and it returns `embed-multilingual-v2.0` / 768 without the
change. ruff check and format are clean on the touched Python files. No
Cohere request was made.

This change was made with Claude Code.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-29 00:16:28 +08:00
Brendan Clement 4359688f32 ci: let the Vale action install its pinned CLI (#4708) 
Let the pinned Vale action install Vale 3.24.0 instead of maintaining a
separate download and extraction step
 2026-10-28 23:26:40 +08:00
Brendan Clement 149cf1052a fix(ci): bundle Vale rules to avoid private release downloads (#4707) 
The prose check fails because it downloads its Vale rules from the
now-private docs repository.

Keep the five unchanged rules from `vale-lancedb-v0.1.0` in this
repository and disable `vale sync` in CI. This also works for
contributors without access to the docs repository. The update
instructions record the release and archive checksum.
 2026-10-28 08:16:42 -07:00
Lance Release 45f987a29b Bump version: 0.41.0-beta.15 → 0.41.0-beta.16 2026-10-28 05:01:52 +00:00
Wyatt Alt bb6aa1a147 feat: add a cascade option to drop_database (#4697) 
The remote catalog could only drop an empty database: it always asked
the server for a restricted drop.

`DropDatabaseRequest::cascade`, `drop_database(..., cascade=True)` in
Python and `dropDatabase(name, { cascade: true })` in TypeScript ask the
server to drop everything in the database first. The name stays taken
until the server finishes cleaning up the dropped objects' storage. A
restricted drop remains the default.
 2026-10-27 21:51:10 -07:00
Jack YeandXuanwo 45dbf5eff5 fix: use SQL for all remote materialized view APIs (#4673) 
Remote materialized-view APIs now use SQL for the full lifecycle. Create
and refresh already submit SQL; this change moves describe, paginated
listing, and drop onto `DESCRIBE MATERIALIZED VIEW`, `SHOW MATERIALIZED
VIEWS`, and `DROP MATERIALIZED VIEW`.

Async create, refresh, and drop return the existing Job handle with
`job_type = "query"`. The handle has one `id`: the connection-scoped SQL
query ID, also accepted by `describe_query`. Submission starts the
query, the job drains its result once, and repeated waits reuse the
completed result. Synchronous drop waits for catalog removal; physical
cleanup proceeds in the background.

Listing uses namespace-scoped keyset pagination with opaque resume
tokens. Streamed and deferred SQL responses share the same path. Adds
transport coverage and updates Python/TypeScript documentation for
query-backed drop jobs.

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-28 04:37:00 +08:00
Jay PokaleandXuanwo f6a348ac5b fix(python): keep FTS scores in RRF and MRR rerankers with return_score="all" (#4677) 
`RRFReranker.rerank_hybrid` and `MRRReranker.rerank_hybrid` merge the
vector and FTS results with the base `merge_results`, which keeps only
the first copy of a duplicated row. With `return_score="all"`, a row
that both searches found therefore comes back with `_score = None`, even
though the FTS search scored it. `LinearCombinationReranker` already
copies the FTS score onto such rows.

Reproduction against `lancedb` 0.40.0:

```python
import pyarrow as pa
from lancedb.rerankers import RRFReranker

vector = pa.table({"_rowid": pa.array([1, 2], pa.uint64()), "text": ["a", "b"],
                   "_distance": pa.array([0.1, 0.2], pa.float32())})
fts = pa.table({"_rowid": pa.array([2, 3], pa.uint64()), "text": ["b", "c"],
                "_score": pa.array([5.0, 4.0], pa.float32())})
RRFReranker(return_score="all").rerank_hybrid("q", vector, fts).to_pylist()
# row 2: _distance=0.2, _score=None   (expected _score=5.0)
```

`table.search(query_type="hybrid")` is not affected, because
`_combine_hybrid_results` restores the original `_distance` and `_score`
by `_rowid` after reranking. The wrong value shows up when
`rerank_hybrid` is called directly, for example from a custom reranker
that wraps RRF or MRR.

The fix uses `_merge_and_keep_scores` when `return_score="all"`, the
same pattern the model-based rerankers (`CohereReranker`,
`CrossEncoderReranker`, `JinaReranker`, `OpenaiReranker`,
`VoyageAIReranker`, ...) already use. The relevance-only path is
unchanged.

Tests: added `test_rank_fusion_return_score_all_keeps_both_scores` for
both rerankers. It fails without the fix and passes with it. I ran it
against the released 0.40.0 package with this branch's
`rerankers/{base,rrf,mrr}.py` dropped in; `rrf.py` and `mrr.py` on
`main` are identical to 0.40.0. The other reranker tests give the same
results with and without the change. `ruff format` and `ruff check` are
clean.

I used an AI assistant (Claude) while working on this; I reviewed and
tested the change.

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-28 03:33:36 +08:00
lancedb-gatefixer[bot]andXuanwo ac99ab173b fix(python): support native blob conversion in pandas and batches (#4647) 
Plain local blob v2 scans tried to open a pylance dataset before
reaching LanceDB's byte-fetch path, so `to_pandas(blob_mode="bytes")`
failed when the optional dependency was absent. Batch readers also had
no `blob_mode` option.

Route supported blob v2 pandas conversions through the native Arrow
path. Add `blob_mode` to sync and async batch readers, convert payloads
during iteration, and expose the converted schema without fetching the
first batch. Apply conversion after reranking and retain the default
descriptor behavior. Clean up scanner error punctuation and remove the
misleading plain-scan suggestion.

Bytes mode retains the existing `to_arrow` restrictions for legacy/list
blobs, remote tables, and namespace query pushdown. Lazy pandas handles
and legacy pandas scans still require pylance; sync FTS/hybrid batching
remains unsupported.

Validation:
- UV-managed editable Python development build completed successfully.
- The absent-pylance regression tests fail against the original query
implementation and pass with this change.
- Blob and query suites: 403 passed, 1 skipped, 1 deselected.
- Remote query suite: 190 passed.
- Python-tree Ruff lint/format checks and the async reader Pyright check
pass. Repository-wide Ruff reports existing errors outside the Python
tree.

Fixes #4645

<!-- lance-gatekeeper-fix:v1 agent=fd85998f14bb0b3c92b2d050df93a615
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-28 02:59:02 +08:00
Dan Tasse 6e4f523e14 docs: remove logical-column from docs (#4693) 
We've removed this feature from LanceDB Enterprise and no longer
privilege the "lancedb:logical-column" key. So, removing that from these
docs.
 2026-10-27 13:59:51 -04:00
Xuanwo b44a0d2362 chore: update Lance to v15.0.0-beta.3 (#4692) 
Upgrade Lance to 15.0.0-beta.3 so `Dataset::concat_data_file_parts`
returns a `Send` future and downstream workers can run it on Tokio. The
upstream fix is https://github.com/lance-format/lance/pull/10370.

Keep the vendored Arrow validity fix; upstream Arrow sources are
unchanged between beta.2 and beta.3.
 2026-10-28 01:49:32 +08:00
dependabot[bot] d44609b6f2 chore(deps): bump the nodejs-deps group across 1 directory with 3 updates (#4661) 
Bumps the nodejs-deps group with 3 updates in the /nodejs directory:
[@opentelemetry/api](https://github.com/open-telemetry/opentelemetry-js),
[@opentelemetry/sdk-metrics](https://github.com/open-telemetry/opentelemetry-js)
and [ts-jest](https://github.com/kulshekhar/ts-jest).

Updates `@opentelemetry/api` from 1.9.1 to 1.10.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/open-telemetry/opentelemetry-js/releases">@​opentelemetry/api's
releases</a>.</em></p>
<blockquote>
<h2>api/v1.10.0</h2>
<h2>1.10.0</h2>
<h3>🚀 Features</h3>
<ul>
<li>feat(api): integrate the Logs API into
<code>@opentelemetry/api</code> <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/4862">#4862</a>
<a
href="https://github.com/hectorhdzg"><code>@​hectorhdzg</code></a></li>
<li>feat(api): add imperative <code>context.attach()</code> for setting
context across callback boundaries that <code>with()</code> cannot wrap.
<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6845">#6845</a>
<a href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a>
<ul>
<li><code>attach()</code> returns a <code>Token</code> whose
<code>dispose()</code> method restores the previous context. Use
<code>token.dispose()</code> to detach.</li>
<li><code>attach</code> is an optional method on the
<code>ContextManager</code> interface; when the active context manager
does not implement it, <code>context.attach()</code> logs a warning and
returns a no-op token.</li>
</ul>
</li>
<li>feat(api): add <code>AnyValue</code> type (of type
<code>unknown</code>) and extend <code>Attributes</code> type values to
use <code>AnyValue</code> <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6780">#6780</a>
<a href="https://github.com/trentm"><code>@​trentm</code></a>
<ul>
<li>This is part of supporting <a
href="https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/4485-extending-attributes-to-support-complex-values.md#how">OTEP
4485</a>.</li>
<li>This is a <strong>breaking change for SDK implementations</strong>,
in that they may need to support more attribute types for any APIs that
accept <code>Attributes</code> or <code>AnyValue</code>. Implementors
should refer to the OTEP 4485 link above for &quot;MUST / SHOULD&quot;
requirements for complex attributes handling. See also the <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/packages/opentelemetry-core/src/common/attributes.ts">attributes-handling
utilities provided by <code>@opentelemetry/core</code> to help</a>. For
example, <code>cleanSimpleAttributes()</code> can be used to limit
user-provided attributes to the set of value types supported in earlier
<code>@opentelemetry/api</code> versions.</li>
<li>This change is <strong>typically not a breaking change for
users</strong> of <code>@opentelemetry/api</code>. There is one possible
breaking case: If a TypeScript user was using the
<code>Attributes</code> type in their own code and using <a
href="https://www.typescriptlang.org/docs/handbook/2/narrowing.html">TypeScript
Narrowing</a> to reduce an attribute value to an expected type (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6579#issuecomment-4345316200">example
here</a>). It is expected that this usage should be rare and, likely,
limited to SDK implementations, if at all. (If this impacted your code,
<a
href="https://github.com/open-telemetry/opentelemetry-js/issues/new">please
let us know</a>.)</li>
<li><strong>Note for users of attributes:</strong> Simple attributes
SHOULD be used whenever possible. Instrumentations SHOULD assume that
backends do not index individual properties of complex attributes, that
querying or aggregating on such properties is inefficient and
complicated, and that reporting complex attributes carries higher
performance overhead.</li>
</ul>
</li>
</ul>
<h3>🐛 Bug Fixes</h3>
<ul>
<li>fix(api): cap tracestate list-members and total length in the
deprecated <code>TraceState.set()</code> so it stays in step with
<code>TraceState</code> from <code>@opentelemetry/core</code> <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6964">#6964</a>
<a href="https://github.com/Zuhef"><code>@​Zuhef</code></a></li>
<li>fix: resolve the ESM build through the <code>default</code> export
condition instead of <code>import</code>, so a resolver that activates
neither <code>import</code> nor <code>require</code> still finds an
entry point <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7132">#7132</a>
<a
href="https://github.com/overbalance"><code>@​overbalance</code></a></li>
</ul>
<h3>📚 Documentation</h3>
<ul>
<li>docs(api): complete <code>@since</code> annotations for the public
API <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7023">#7023</a>
<a
href="https://github.com/nabeelamjadsheikh"><code>@​nabeelamjadsheikh</code></a>
<ul>
<li>Adds the three missing exported-type annotations and the
member-level annotations for methods and properties introduced after
their enclosing type.</li>
<li>Corrects <code>Attributes</code>, <code>AttributeValue</code> and
<code>TracerOptions</code> from <code>1.3.0</code> to
<code>1.1.0</code>; all three were already importable in 1.1.0 via
wildcard re-export.</li>
</ul>
</li>
</ul>
<h3>🏠 Internal</h3>
<ul>
<li>build: migrate the package build from <code>tsc</code> to
<code>tsdown</code>, emitting dual CJS/ESM output from a single
<code>dist/</code> directory <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6293">#6293</a>
<a href="https://github.com/overbalance"><code>@​overbalance</code></a>
<ul>
<li>The public entry points are unchanged:
<code>@opentelemetry/api</code> and
<code>@opentelemetry/api/experimental</code>. This package already
declared an <code>exports</code> map, so deep imports such as
<code>@opentelemetry/api/build/src/...</code> were rejected before this
change and continue to be.</li>
<li>The emitted files moved out of <code>build/src</code> (CJS),
<code>build/esm</code> and <code>build/esnext</code> (ESM) into
<code>dist/</code>, using <code>.cjs</code>/<code>.mjs</code> extensions
with matching <code>.d.cts</code>/<code>.d.mts</code> declarations. The
non-standard <code>esnext</code> condition has been dropped in favour of
the standard <code>import</code>/<code>require</code> conditions.</li>
</ul>
</li>
<li>perf(api): add getGlobal fast-path <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6956">#6956</a>
<a
href="https://github.com/legendecas"><code>@​legendecas</code></a></li>
<li>chore: remove the redundant <code>module</code> field from
<code>package.json</code>; bundlers resolve the ESM build through the
<code>exports</code> map <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7130">#7130</a>
<a
href="https://github.com/overbalance"><code>@​overbalance</code></a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md">@​opentelemetry/api's
changelog</a>.</em></p>
<blockquote>
<h2>1.10.0</h2>
<h3>🚀 (Enhancement)</h3>
<ul>
<li>feat(resource): create sync resource with some attributes that
resolve asynchronously <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3460">#3460</a>
<a
href="https://github.com/samimusallam"><code>@​samimusallam</code></a></li>
<li>feat: collect host id for non-cloud environments <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3575">#3575</a>
<a href="https://github.com/mwear"><code>@​mwear</code></a></li>
<li>feat (api-logs): separate Events API into its own package <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3550">3550</a>
<a
href="https://github.com/martinkuba"><code>@​martinkuba</code></a></li>
<li>feat(sdk-metrics): apply binary search in histogram recording <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3539">#3539</a>
<a
href="https://github.com/legendecas"><code>@​legendecas</code></a></li>
<li>perf(propagator-jaeger): improve deserializeSpanContext performance
<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3541">#3541</a>
<a href="https://github.com/doochik"><code>@​doochik</code></a></li>
<li>feat: support TraceState in SamplingResult <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3530">#3530</a>
<a
href="https://github.com/raphael-theriault-swi"><code>@​raphael-theriault-swi</code></a></li>
<li>feat(sdk-trace-base): add diagnostic logging when spans are dropped
<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3610">#3610</a>
<a
href="https://github.com/neoeinstein"><code>@​neoeinstein</code></a></li>
<li>feat: add unit to view instrument selection criteria <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3647">#3647</a>
<a href="https://github.com/jlabatut"><code>@​jlabatut</code></a></li>
<li>feat(tracing): expose dropped counts for attributes, events and
links on span <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3576">#3576</a>
<a href="https://github.com/mohitk05"><code>@​mohitk05</code></a></li>
</ul>
<h3>🐛 (Bug Fix)</h3>
<ul>
<li>fix(core): added falsy check to make otel core work with browser
where webpack config had process as false or null <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3613">#3613</a>
<a
href="https://github.com/ravindra-dyte"><code>@​ravindra-dyte</code></a></li>
<li>fix(instrumentation-http): include query params in http.target <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3646">#3646</a>
<a href="https://github.com/kobi-co"><code>@​kobi-co</code></a></li>
</ul>
<h3>📚 (Refine Doc)</h3>
<ul>
<li>chore: update http example <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3651">#3651</a>
<a
href="https://github.com/JamieDanielson"><code>@​JamieDanielson</code></a></li>
</ul>
<h3>🏠 (Internal)</h3>
<ul>
<li>chore(exporter-jaeger): deprecate jaeger exporter <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3585">#3585</a>
<a
href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a></li>
<li>fix(sdk-metrics): fix flaky LastValueAggregator test by using fake
timer <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3587">#3587</a>
<a
href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a></li>
<li>fix(test): fix failing tests by preventing source-map generation <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3642">#3642</a>
<a
href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/56e6b1bb890f844b8963a146780d0b9cfa8abd0d"><code>56e6b1b</code></a>
chore: Release API 1.4.1, SDK 1.10.0, Experimental 0.36.0 (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3654">#3654</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/ecb5ebe86eebc5a1880041b2523adf5f6d022282"><code>ecb5ebe</code></a>
fix: Minor typing issues for web sandbox (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3660">#3660</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/48ecf229eb0b468cf433a152daf414313b381edc"><code>48ecf22</code></a>
feat(sdk-trace-base): add dropped attributes and events count on span
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3576">#3576</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/95bf6fc7cc105469def055f1f93446c9cd09882d"><code>95bf6fc</code></a>
fix(build): add <code>example/http</code> to ignore list in
<code>update-ts-configs</code> (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3657">#3657</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/3a2ca58075cc7e0eea8233c51735db5d583c11d0"><code>3a2ca58</code></a>
feat: add unit to view instrument selection criteria (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3647">#3647</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/ebc8575a0c215a7b7c84866116a261ffd8aff015"><code>ebc8575</code></a>
chore(jeager-exporter): deprecate jaeger exporter (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3585">#3585</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/2f715bd78dd7fb66e2aa2b76fb6e7c623deddd60"><code>2f715bd</code></a>
feat: allow to add attributes from grpc metadata in the patched server
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3589">#3589</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/cbbdfd379854fcb41bd70841ae4890200a2fda09"><code>cbbdfd3</code></a>
fix(sdk-node): update instrumentations once MeterProvider is initialized
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3624">#3624</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/4a4484a910201466121ed45fa12566737f6ac19d"><code>4a4484a</code></a>
chore: update http example (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3651">#3651</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/494efffcf4648e3b1207c3c925c1a52bcc7f7350"><code>494efff</code></a>
fix(otlp-exporter-base): fix changelog for merged retry pr (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3650">#3650</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/open-telemetry/opentelemetry-js/compare/v1.9.1...v1.10.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `@opentelemetry/sdk-metrics` from 2.11.0 to 2.12.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/open-telemetry/opentelemetry-js/releases">@​opentelemetry/sdk-metrics's
releases</a>.</em></p>
<blockquote>
<h2>v2.12.0</h2>
<h2>2.12.0</h2>
<h3>🚀 Features</h3>
<ul>
<li>chore(sdk-trace): add <code>StackContextManager</code> in
<code>@opentelemetry/sdk-trace</code> package <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7086">#7086</a></li>
</ul>
<h3>🏠 Internal</h3>
<ul>
<li>feat(ci): support releasing from maintenance branches <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6767">#6767</a>
<a href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a>
<ul>
<li>The API documentation site is only redeployed for releases whose
commit is reachable from <code>main</code>, so a maintenance release no
longer overwrites it.</li>
</ul>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md">@​opentelemetry/sdk-metrics's
changelog</a>.</em></p>
<blockquote>
<!-- raw HTML omitted -->
<h1>CHANGELOG</h1>
<p>All notable changes to this project will be documented in this
file.</p>
<p>For API changes, see the <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/api/CHANGELOG.md">API
CHANGELOG</a>.
For experimental package changes, see the <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/experimental/CHANGELOG.md">experimental
CHANGELOG</a>.
For semantic convention package changes, see the <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/semantic-conventions/CHANGELOG.md">semconv
CHANGELOG</a>.
For notes on migrating to 2.x / 0.200.x see <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/upgrade-to-2.x.md">the
upgrade guide</a>.
For notes on migrating to 3.x see <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/3.x/migration-guide.md">the
3.x migration guide</a>.</p>
<h2>Unreleased</h2>
<h3>💥 Breaking Changes</h3>
<h3>🚀 Features</h3>
<h3>🐛 Bug Fixes</h3>
<h3>📚 Documentation</h3>
<h3>🏠 Internal</h3>
<h2>3.0.0</h2>
<h3>💥 Breaking Changes</h3>
<ul>
<li>chore(sdk-trace-node)!: remove the
<code>@opentelemetry/sdk-trace-node</code> package <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7054">#7054</a>
<ul>
<li>The sdk-trace-node package has been replaced by the
<code>@opentelemetry/sdk-trace</code> package.
See the <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/3.x/migration-guide.md">3.x
migration guide</a> for full migration instructions.</li>
</ul>
</li>
<li>chore(shim-opentracing)!: remove the
<code>@opentelemetry/shim-opentracing</code> package
<ul>
<li>In the <a
href="https://github.com/open-telemetry/opentelemetry-specification/releases/tag/v1.56.0">OpenTelemetry
Specification v1.56.0</a> the <a
href="https://redirect.github.com/open-telemetry/opentelemetry-specification/pull/4938">OpenTracing
compatibility requirements were deprecated</a>. The JavaScript
OpenTracing shim package will not receive any more releases after the
current <a
href="https://www.npmjs.com/package/@opentelemetry/shim-opentracing/v/2.12.0">2.12.0
release</a> (<a
href="https://github.com/open-telemetry/opentelemetry-js/tree/v2.12.0/packages/opentelemetry-shim-opentracing/">source
code for last release</a>).</li>
</ul>
</li>
<li>chore(context-async-hooks)!: remove the unused class
<code>AsyncHooksContextManager</code> <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7078">#7078</a></li>
<li>feat!: migrate package builds from <code>tsc</code> to
<code>tsdown</code>, emitting dual CJS/ESM output from a single
<code>dist/</code> directory and declaring an <code>exports</code> map
on every package <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6293">#6293</a>
<a href="https://github.com/overbalance"><code>@​overbalance</code></a>
<ul>
<li>Importing a package by its name is unaffected in both CommonJS and
ESM, as is every subpath listed in its <code>exports</code> map.</li>
<li><strong>Deep imports into the build output no longer
resolve.</strong> An <code>exports</code> map is an allowlist that
Node.js and bundlers enforce, so specifiers such as
<code>@opentelemetry/core/build/src/...</code> or
<code>@opentelemetry/core/build/esm/...</code> now fail with
<code>ERR_PACKAGE_PATH_NOT_EXPORTED</code>. Rewriting them to the new
file layout does not help — unlisted subpaths are rejected whether or
not the file exists.</li>
<li>The emitted files moved out of <code>build/src</code> (CJS),
<code>build/esm</code> and <code>build/esnext</code> (ESM) into
<code>dist/</code>, using <code>.cjs</code>/<code>.mjs</code> extensions
with matching <code>.d.cts</code>/<code>.d.mts</code> declarations.</li>
<li><code>&lt;package&gt;/package.json</code> is no longer importable
for the same reason. Read a package's version from your own dependency
metadata, or use <code>SDK_INFO</code> from
<code>@opentelemetry/core</code> for the SDK version.</li>
<li>The non-standard <code>esnext</code> entry point has been removed;
tools that preferred it fall back to <code>module</code> (ESM) or
<code>main</code> (CJS).</li>
<li>If you depend on something that is only reachable through a deep
import, please open an issue so it can be considered for the public
API.</li>
</ul>
</li>
<li>feat(core)!: remove deprecated <code>getTimeOrigin</code>,
<code>otperformance</code>, <code>_globalThis</code>, and
<code>unrefTimer</code> from <code>@opentelemetry/core</code> <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7053">#7053</a>
<ul>
<li><code>getTimeOrigin()</code> — use
<code>performance.timeOrigin</code> directly.</li>
<li><code>otperformance</code> — use the global <code>performance</code>
object directly.</li>
<li><code>_globalThis</code> — use <code>globalThis</code>
directly.</li>
<li><code>unrefTimer(timer)</code> — call <code>timer.unref()</code>
directly in your own code.</li>
</ul>
</li>
<li>feat(sdk-trace)!: remove deprecated
<code>TracerProviderOptions.forceFlushTimeoutMillis</code> <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7057">#7057</a>
<ul>
<li>Pass <code>timeoutMillis</code> to <code>provider.forceFlush({
timeoutMillis })</code> instead. The default timeout is 30000ms.</li>
</ul>
</li>
<li>feat(propagator-jaeger)!: remove
<code>@opentelemetry/propagator-jaeger</code> package <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7077">#7077</a>
<ul>
<li>The Jaeger propagator is deprecated by the OpenTelemetry
specification in favour of <code>W3CTraceContextPropagator</code>. Use
<code>W3CTraceContextPropagator</code> from
<code>@opentelemetry/core</code> instead.</li>
<li>See the <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/3.x/migration-guide.md">3.x
migration guide</a> for full instructions.</li>
</ul>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/92b844e8aa7d9655e4b3560154e912c4c1ced64d"><code>92b844e</code></a>
chore: prepare next v2.x release (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7151">#7151</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/a859aff9825ec8138cf287d08ac86401f00aad7c"><code>a859aff</code></a>
fix(web-common): export getElementXPath utility (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7124">#7124</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/2bd5a73e7af7ab363f3b239f8cb1f5f56eaa3ad5"><code>2bd5a73</code></a>
chore(web-common)!: deprecate <code>addSpanNetworkEvents()</code> et al
instrumentation-...</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/d60f05edeac6aec8e64736c10672077fd4a766b5"><code>d60f05e</code></a>
doc(sdk-trace): add description for StackContextManager (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7089">#7089</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/5bd476e6f7a22e6cf2e94a645833247284aa295d"><code>5bd476e</code></a>
chore: move context manager to sdk-trace (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7086">#7086</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/554c0c6467c77e9ef1f2676f2bd20b505e228417"><code>554c0c6</code></a>
chore: move sdk-trace-web utils to web-common (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7058">#7058</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/788922c5780a17d5aaace76a0da458858b2ad88a"><code>788922c</code></a>
ci: support maintenance branch releases (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7036">#7036</a>)</li>
<li>See full diff in <a
href="https://github.com/open-telemetry/opentelemetry-js/compare/v2.11.0...v2.12.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `ts-jest` from 29.4.12 to 29.4.15
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/kulshekhar/ts-jest/releases">ts-jest's
releases</a>.</em></p>
<blockquote>
<h2>v29.4.15</h2>
<p>Please refer to <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v29.4.14</h2>
<p>Please refer to <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v29.4.13</h2>
<p>Please refer to <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">ts-jest's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.14...v29.4.15">29.4.15</a>
(2026-10-15)</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>compiler:</strong> resolve modules with tsconfig options
regardless of cache state (<a
href="https://github.com/kulshekhar/ts-jest/commit/268746c0fcaf72476cebb348c5543e5c2aac3622">268746c</a>),
closes <a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/5216">#5216</a></li>
<li>support <code>.cjs</code> config files in
<code>config:migrate</code> (<a
href="https://github.com/kulshekhar/ts-jest/commit/4001b4e6d28aa461e7468f09157413e113f0bd92">4001b4e</a>),
closes <a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/4485">#4485</a></li>
</ul>
<h2><a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.13...v29.4.14">29.4.14</a>
(2026-09-25)</h2>
<h3>Reverts</h3>
<ul>
<li>bring back prior changes of compiler util (<a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/5471">#5471</a>)
(<a
href="https://github.com/kulshekhar/ts-jest/commit/01e5a2e17ac9c940880865d6a81751832dcc4184">01e5a2e</a>),
closes <a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/5469">#5469</a></li>
</ul>
<h2><a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.12...v29.4.13">29.4.13</a>
(2026-09-22)</h2>
<h3>Features</h3>
<ul>
<li>support Babel 8 (<a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/5466">#5466</a>)
(<a
href="https://github.com/kulshekhar/ts-jest/commit/c3a0b57c00d8b6acfb72b020e2411752d39073e2">c3a0b57</a>),
closes <a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/5412">#5412</a></li>
</ul>
<h3>Performance Improvements</h3>
<ul>
<li><strong>compiler:</strong> cache module resolution modes (<a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/5419">#5419</a>)
(<a
href="https://github.com/kulshekhar/ts-jest/commit/697d337c1b79a0804972a3d30c0875b02e73bf62">697d337</a>),
closes <a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/5402">#5402</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/197d4de96002589658d8a6c2d3b9b9554d17af6b"><code>197d4de</code></a>
chore(release): 29.4.15</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/ebad800d2c080cb72d7f0ea3225221b0cee066da"><code>ebad800</code></a>
build(deps): update dependency node to v24.22.0</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/e5f6e986fe7c9e8115a7afe49ec9d5e8a576e93b"><code>e5f6e98</code></a>
test(compiler): clean up temp dirs from module resolution tests</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/77dbb184d5895f8e315d574971fc83ab6a8a43b2"><code>77dbb18</code></a>
test(compiler): make module resolution tests path-independent on
Windows</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/268746c0fcaf72476cebb348c5543e5c2aac3622"><code>268746c</code></a>
fix(compiler): resolve modules with tsconfig options regardless of cache
state</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/15362016c490ae230c3d69f7a9399513d3600ed7"><code>1536201</code></a>
build(deps): update dependency vite to ^8.3.4</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/141e39c092759ca6173095fe2dd4ebcdd3117e62"><code>141e39c</code></a>
build(deps): update dependency <code>@​babel/core</code> to ^8.0.7</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/6a6686e5374525db09ac67f37cf0f2d53c372f28"><code>6a6686e</code></a>
build(deps-dev): bump shell-quote from 1.10.0 to 1.12.0 in
/examples</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/98e6e50fe934e8e86c903719048891a3a071a2b7"><code>98e6e50</code></a>
build(deps): bump proxy-addr from 2.0.7 to 2.0.8 in /website</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/d2439ff5ed9f6425bed60d0202f37771022f0526"><code>d2439ff</code></a>
build(deps): bump compression from 1.8.1 to 1.8.2 in /website</li>
<li>Additional commits viewable in <a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.12...v29.4.15">compare
view</a></li>
</ul>
</details>
<br />

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2026-10-27 10:28:51 -07:00
dependabot[bot]Will JonesClaude Opus 5.5
 		0cccc9c991 chore(deps): bump rust-minor-patch group and napi 3.15 (#4669) 
Bumps the rust-minor-patch group: serde_json 1.0.154, tokio 1.54.1, uuid
1.28.0, reqwest 0.13.5, webbrowser 1.2.5, napi-derive 3.6.13, napi-build
2.6.0, arc-swap 1.9.3, libc 0.2.191.

On top of the Dependabot update:

* napi is bumped to 3.15.0 instead of 3.13.0. napi-derive 3.6.13
generates code that references
`bindgen_prelude::NativeBorrowScopeRelease`, which napi 3.13.0 doesn't
have, so the nodejs crate failed to compile. The napi and napi-derive
floors in `nodejs/Cargo.toml` are raised so they can't drift apart
again.
* The RUSTSEC-2026-0410 ignore from #4683 is removed from `deny.toml`.
#4690 already moved the lockfile to zstd-sys 2.1.2, which fixes it. The
other three zstd ignores stay until we can move to Arrow 59+.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Will Jones <willjones127@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-27 10:28:07 -07:00
lancedb-gatefixer[bot] fe079eb421 fix!: restore blob SQL inserts with explicit adapter schemas (#4687) 2026-10-27 09:31:23 -07:00
Ozawa SongandXuanwo 4b373b2070 fix(python)!: support compat-only imports (#3973) 
Read `__version__` from the native `_lancedb` module so `import lancedb`
works when only `lancedb-compat` is installed. Add the type declaration
and a subprocess regression test.

Ruff and Linux-targeted Pyright pass. Import tests: 1 passed and 1
Linux-only test skipped on Windows. Runtime tests used the official
same-version Windows native module; a fresh source build still needs CI.

BREAKING CHANGE: `lancedb.__version__` now uses the native version
string.
Prereleases may report `0.41.0-beta.13` instead of `0.41.0b13`.

Fixes #3950

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-28 00:29:22 +08:00
Jay PokaleandXuanwo dcac834233 fix(python)!: reject tuple fields with mixed element types in pydantic models (#4679) 
`pydantic_to_schema` maps a `Tuple[...]` field to an Arrow list of its
**first** element type. A tuple with different element types gets the
wrong schema, and the error only surfaces when a row is written:

```python
class Pair(LanceModel):
    pair: Tuple[int, str]

pydantic_to_schema(Pair)        # pair: list<item: int64>
db.create_table("t", schema=Pair).add([Pair(pair=(1, "a"))])
# pyarrow.lib.ArrowInvalid: Could not convert 'a' with type str: tried to convert to int64
```

An Arrow list has a single element type, so such a tuple cannot be
stored as one. This raises a `TypeError` when the schema is built,
saying the tuple elements must share one type and pointing to a nested
`LanceModel` for a fixed set of differently typed fields. Homogeneous
tuples are unchanged: `Tuple[int, int]` and `Tuple[float, ...]` still
map to lists.

**Breaking change:** a model with a mixed-type tuple field, which used
to get a (wrong) `list<...>` schema from `pydantic_to_schema`, now
raises `TypeError` when the schema is built. Such a schema could not
hold the model's values: writing a row failed with `ArrowInvalid`. The
requirement and the nested-`LanceModel` alternative are documented in
the `pydantic_to_schema` docstring.

Tests (in `test_pydantic.py`): `TypeError` for `Tuple[int, str]` (fails
on `main`), and the list mapping for `Tuple[int, int]` and `Tuple[float,
...]`. The whole file passes; `ruff format` and `ruff check` are clean.

I used an AI assistant (Claude) while working on this; I reviewed and
tested the change.

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 23:35:04 +08:00
陈志谦andXuanwo 3575f972ac fix(python): accept tuples and pyarrow arrays as hybrid search vectors (#4662) 
The synchronous hybrid query builder rejected vector containers that the
rest of the Python API accepts:

```python
table.search(query_type="hybrid").vector((1.0, 2.0)).text("hello").to_pandas()   # ValueError
table.search(query_type="hybrid").vector(pa.array([1.0, 2.0], pa.float32()))...
# ValueError: Vector query must be either a string or a vector
```

`lancedb.common.VEC` documents `list[float] | tuple[float, ...] |
np.ndarray | pa.Array`, and the same two containers work for
`table.search(vector)` and for the async hybrid API
(`AsyncQuery.nearest_to(...).nearest_to_text(...)`).
`LanceHybridQueryBuilder._validate_query` only listed `str | list |
np.ndarray`, so a tuple or a pyarrow array was refused before reaching
the query planner.

This normalises the vector exactly the way `LanceQueryBuilder.create`
already does — list/tuple to a `np.float32` array, `np.ndarray` through
`astype(float32)`, `pa.Array`/`pa.ChunkedArray` through `np.array(...,
dtype=float32)`, strings passed through — and keeps the existing
`ValueError` for anything else.


`python/python/tests/test_hybrid_query.py::test_hybrid_query_accepts_every_vector_container`
is parametrised over tuple, pyarrow array and pyarrow chunked array, and
asserts the returned `_rowid`s match the `list` form: 3 failed on `main`
(`ValueError` from `query.py:2452`), 3 passed with the change.
Neighbouring suites (`test_hybrid_query.py`, `test_query.py`,
`test_rerankers.py`) are 220 passed / 9 skipped, and `test_table.py`,
`test_remote_db.py`, `test_fts.py`, `test_index.py`,
`test_permutation.py` are 724 passed / 5 skipped. `ruff` at the version
pinned in the repository's pre-commit config (0.9.9) reports no findings
and `ruff format --check` is clean.

One source file (`python/python/lancedb/query.py`), no behaviour change
for containers that already worked.

Prepared with AI assistance; every line was verified by running the
cases above.

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 22:49:31 +08:00
lancedb-gatefixer[bot]andXuanwo 2cd635d41c fix: correct remote materialized view refresh results and options (#4672) 
Remote materialized-view refreshes drained the SQL result and then
fabricated `mode=rebuild`, `rows_written=<total view rows>`, and
`source_version=0`. This made incremental refreshes report incorrect
statistics.

Make these three result fields optional across Rust, Python, and
TypeScript, and leave them unset for remote SQL refreshes. Remove the
post-refresh row count request. Local refreshes continue to populate all
metrics, and `version` remains the view version observed after
completion. Remote per-operation statistics remain unavailable until the
SQL job adapter can retrieve them.

Expose the forced-rebuild option through the Rust builder,
synchronous/asynchronous Python refresh and job methods, and TypeScript
refresh. Remote `full=true` sends `WITH (full = true)`; the default lets
the server choose its refresh mode. Local source-version pinning and
remote job cancellation are preserved.

Local refresh regression tests respect the `experimental-functions`
build gate, while remote refresh coverage also runs in default builds.

Validation:

- Rust: 20 materialized-view tests and 21 remote SQL tests passed with
`remote,experimental-functions`; `cargo fmt --all` passed.
- Python: rebuilt the extension in the uv-managed environment. The
default build passed 23 tests with 27 feature-dependent skips. The
opt-in build passed 61 tests with 3 expected default-build-only skips,
covering the materialized-view suite and 14 async query-serialization
cases from current main. Python Ruff lint and formatting checks passed.
- TypeScript: default and opt-in native builds and TypeScript
compilation passed. The default build passed 5 tests with 10 expected
feature-dependent skips; the opt-in build passed all 14
materialized-view tests with the default-build-only test skipped.
Lint/format checks and documentation generation passed.
- Repository-wide `ruff check .` reports 48 pre-existing violations in
unchanged `docs/web-tests` files; `ruff check python` passes.

Fixes #4670

<!-- lance-gatekeeper-fix:v1 agent=ece43ec4b88a48d1789f60005fa3ecfc
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 21:34:01 +08:00
lancedb-gatefixer[bot]andXuanwo c167b99fa0 fix: read legacy blob columns through blob APIs (#4649) 
Legacy blob v1 columns were rejected by LanceDB's blob validation and
omitted from `blob_columns()`, even though the pinned Lance reader
already supports them. Accept both blob versions in read validation and
discovery so `fetch_blobs`, `fetch_blob_files`, and `fetch_blob_ranges`
can read legacy payloads through the existing native reader.

Keep list-nested blobs excluded from row-based reads and document how to
fetch legacy blobs using explicit row IDs. Regression coverage checks
sync and async APIs, independently written Lance datasets, ordering and
duplicates across fragments, null and empty blobs, lazy seeking, ranges,
and deleted rows.

Fixes #4644

Validation:

- Both new Python regressions failed on the original implementation with
the reported legacy-column rejection and empty discovery result, then
passed with the fix.
- Python blob suite: 202 passed, 1 skipped.
- Remote blob tests: 29 passed.
- Focused Rust blob unit tests with `remote`: 13 passed.
- Rust blob integration suite with `remote`: 59 passed, including legacy
storage 2.0 and 2.1.
- Rust compile and Clippy checks with `remote`, tests, and examples
passed.
- Python package Ruff lint and format checks passed; Rust formatting
passed.
- Repository-wide `ruff check .` reports 49 existing errors in unchanged
`docs/web-tests` examples; the Python package is clean.

<!-- lance-gatekeeper-fix:v1 agent=02b7edcbfa8ff0e7fad4615159249b3e
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 19:56:50 +08:00
lancedb-gatefixer[bot]andXuanwo 83b7275137 fix(remote): make HTTP timeout errors actionable (#4650) 
Remote `create_table` reader uploads can exceed the read deadline while
waiting for response headers, but reqwest's main error message only says
"error sending request". Make HTTP and retry errors explicitly identify
timeouts and name the relevant `client_config.timeout_config` settings
and environment variables, preserving typed sources, request metadata,
and retry classification.

Document that the read timeout covers waiting for headers during upload.
This implements the issue's diagnostic option: creation still uses one
streaming request, so large or slow readers may require a longer
`client_config.timeout_config.read_timeout`.

Validation:
- 48 focused Rust client, error, and retry tests passed. Regression
coverage holds an upload open until its read timeout and also exercises
TLS connection timeouts, with and without retries.
- 8 Python create-table and HTTP/retry error tests passed in the
uv-managed environment after rebuilding the extension, including reader
creation with read and overall timeouts.
- `cargo check --quiet --features remote --tests --examples` passed.
- `cargo clippy --quiet --features remote --tests --examples` passed.
- Rust formatting and Python package Ruff lint/format checks passed.
- Repository-wide `ruff check .` reports 49 existing findings under
`docs/web-tests`; those files are unchanged.

Fixes #4646

<!-- lance-gatekeeper-fix:v1 agent=ce1c040fa581f1748166ecb686d9c9c0
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 18:51:06 +08:00
Xuanwo ad1212fb32 chore: update Lance to v15.0.0-beta.2 (#4690) 
**Blocked on the corrected Lance release:**
https://github.com/lance-format/lance/pull/10370 restores the `Send`
contract required by downstream Function workers. Refresh this upgrade
with that release before merging.

Upgrade the Rust, Java, and Python Lance dependencies to
https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.2. This
brings in bounded vector-index consolidation without retraining
(https://github.com/lance-format/lance/pull/10335) and deletion
filtering for index entries whose fragment coverage was pruned
(https://github.com/lance-format/lance/pull/10325).

Synchronize the vendored `lance-arrow` with the release and retain its
offset-safe validity patch. The existing `nullable_struct_merge`
regression still panics with the unpatched release, so removing that
patch would regress nullable nested queries. Refresh both lockfiles,
including the `zstd-sys` version required by the new Lance encoding
dependency.

Update the custom memtable bitmap test adapter to train on row
addresses, matching the new bitmap contract; remove its obsolete row-ID
conversion.

Preserve the Blob APIs' existing missing-row error messages when newer
Lance rejects a missing fragment earlier in the read path. Retain
upstream attribution in vendored sources while satisfying the repository
license-header check.

Validation of the compatibility fixes: all 58 Rust Blob integration
tests and 219 Python Blob tests pass (one Python test skipped). Strict
workspace Clippy, formatting, and diff checks pass.
 2026-10-27 17:35:43 +08:00
lancedb-gatefixer[bot]andXuanwo 7fca4d666a fix(python): preserve async query search options during serialization (#4659) 
Async `to_query_object()` dropped enabled `fast_search` on every query
type and `postfilter` on plain and full-text queries. Re-executing a
serialized fast search could therefore return unindexed rows.

Copy enabled `fast_search` into the Python `Query` and serialize enabled
plain/FTS postfilter from the Rust request's prefilter setting. Preserve
the existing unset defaults: `None` for these options, with the vector
query's existing `False` postfilter default.

Validation:

- Added 12 serialization cases across plain, FTS, and vector queries,
covering both enabled and unset options, plus two FTS execution
round-trip cases. Before the fix, nine cases failed, including returning
an unindexed row after serialization.
- Rebuilt the Python extension using the uv-managed environment.
- Query tests and existing FTS fast-search tests: 192 passed, 1 slow
test deselected.
- Rust formatting and Python package lint/format checks pass.
Repository-wide `ruff check .` reports 49 pre-existing errors in
unchanged `docs/web-tests` files.

Fixes #4658

<!-- lance-gatekeeper-fix:v1 agent=f828b24871a2dc928668be2de4f85078
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 16:32:32 +08:00
lancedb-gatefixer[bot]andXuanwo 1edfa6077b fix(python): allow phrase queries without a column (#4653) 
Quoted async full-text queries without `columns` carry an unset phrase
column, but converting them to Python unwrapped it and raised
`PanicException`. Python also required `PhraseQuery.column`, preventing
callers from constructing the equivalent query object.

Make the Python column optional and preserve `Option<String>` in both
Rust conversion directions, leaving column selection to query planning.
Regression tests cover quoted strings and structured queries with
omitted, `None`, and explicit columns, query reuse, JSON serialization,
slop, and document granularity.

Validation:

- Reproduced both missing-column failures before the fix;
explicit-column controls passed.
- Rebuilt the extension with `uv run --extra tests --extra dev maturin
develop --extras tests,dev`.
- `cd python && uv run --no-sync --extra tests pytest
python/tests/test_fts.py
python/tests/test_query.py::test_query_serialization_async -q`: 85
passed.
- `cargo fmt --all -- --check` and `ruff check python` pass; touched
Python files pass formatting checks.
- `cargo clippy --quiet -p lancedb-python --lib --features remote`
passes.
- Root-wide `ruff check .` reports 49 existing findings in unchanged
documentation examples.

Fixes #4651

<!-- lance-gatekeeper-fix:v1 agent=62d36fad70c8717236283b8e97dd6b30
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 15:53:14 +08:00
Avanish GargandXuanwo 8887c82789 fix(python)!: raise on bool(Expr) instead of silently dropping predicates (#4682) 
Python's `and` / `or` / `not` and chained comparisons call `bool()` on
an `Expr`. `Expr` had no
`__bool__`, so that was always `True` and the filter silently lost a
predicate: `where(lit(2) <
col("id") < lit(5))` returned ids 0..4 (lower bound dropped),
`(col("id") > 7) and (col("g") == 0)`
returned all even ids, and `(a) or (b)` kept only `a` (see the issue for
the repro and output).

Changes (`python/python/lancedb/expr.py`,
`python/python/lancedb/query.py`):
- `Expr.__bool__` raises `TypeError("The truth value of an Expr is
ambiguous. ...")` with a hint to
use `&`, `|`, `~` with parentheses and `Expr.isin()` for membership.
`TypeError` rather than
`ValueError` because the object does not support truthiness at all
(polars `Expr` and SQLAlchemy
clauses raise `TypeError`; the codebase raises `TypeError` for
unsupported `lit()` values).
  pandas, numpy and pyarrow `Expression` use `ValueError`.
- `LanceHybridQueryBuilder` tested `if self._where:`, which would have
started raising for every
`.where(Expr)` on a hybrid query. It is now `isinstance(self._where,
Expr) or self._where`, so
an Expr is always applied and an empty SQL string is still skipped as
before.
- Notes section in the `Expr` docstring.

Not changed: no other code in `python/python/lancedb` tests the
truthiness of a filter or of an
`Expr` (checked `where`/`filter`/`predicate`/`condition` handling in
`query.py`, `table.py`,
`remote/`, `merge.py`, `functions.py`, `pydantic.py`: they use `is None`
/ `isinstance`). Cross-SDK:
`nodejs` has no `Expr` class on `main` yet, and JavaScript cannot
overload `&&` / `||` / `!`
either, so there is nothing equivalent to port; the Rust core is not
affected.

Behaviour changes to be aware of: code that already used `and` / `or` /
`not` / chained
comparisons on an `Expr` (which returned wrong rows) now raises
`TypeError`. `x in [expr, ...]`,
`any([...])` and `all([...])` on Exprs, and `==` between two `Query`
objects whose `filter` are
different Expr objects (pydantic compares the fields), now raise instead
of returning a meaningless
answer; `Expr` stays unhashable and `isinstance` / `is None` checks are
unaffected.

Tests (`python/python/tests/test_expr.py`): `TestExprTruthiness`
(`bool`, `and`, `or`, `not`,
chained comparison, `if`, list membership, `any`/`all` raise with a
helpful message; `&`, `|`, `~`
still combine) and `TestExprWhereBuilders` (plain, vector, FTS, hybrid
and async `.where(Expr)`
return the right rows, hybrid with an SQL string and with an empty
string is unchanged, chained
comparison is rejected before any query runs). With the source change
stashed 11 of the new tests
fail; with only `__bool__` added and the hybrid change reverted the
hybrid test fails with the
new `TypeError`, so it guards the second change. `test_expr.py`: 144
passed (123 before this change).

Verification caveat: `main`'s Python layer is newer than the newest
published wheel, so the tests
were run against the 0.41.0b14 native module
(`python/python/lancedb/expr.py` and `query.py` are
identical to the wheel); the full suite (`pytest python/tests
--ignore=python/tests/docs -m "not slow and not s3_test"`)
has the identical 25 failures before and after (wheel/main skew in
remote options, remote db,
index and table tests; none in expr/query tests): 2604 passed before,
2625 passed after (the 21
new tests). `ruff format --check python` and `ruff check python` are
clean. Not run: Rust build, Node tests, remote (cloud)
tests.

Fixes #4681

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 15:14:16 +08:00
Xuanwo 8a4effa06a fix: gate experimental local functions behind an opt-in feature (#4688) 
Published SDK builds currently expose local SQL computed columns and
materialized-view execution that are not ready for release. Add the
opt-in `experimental-functions` Cargo feature to the Rust core and
forward it through the Python and Node bindings; published packages
leave it disabled.

Local declarations and refreshes now return `NotSupported` before
mutating tables or spawning jobs, including prepared materialized-view
creation. Remote Cloud/Enterprise client operations, Python UDF
packaging, ordinary eager `add_columns`, and access to existing data
remain available. Stored metadata and write protections stay active, and
shared server commit helpers remain usable.

CI covers both feature configurations. Existing local execution tests
run with the feature enabled; default-build regressions cover early
rejection, unchanged table versions, existing metadata protections, and
normal eager transforms. API and source-build documentation explain the
opt-in.

Validation: Rust library tests pass with the feature disabled (1,869
tests) and enabled (1,947 tests), plus three default-build integration
regressions. Python and TypeScript bindings were rebuilt and tested in
both configurations, including mocked remote requests; Rust formatting,
Python package lint, TypeScript lint/build, and generated docs were
checked. Repository-wide Ruff still reports 48 pre-existing errors in
unchanged `docs/web-tests` files.
 2026-10-27 14:32:09 +08:00
lancedb-gatefixer[bot] dcbf5ce66a fix: preserve null blob descriptor fields in SQL (#4571) 
SQL descriptor access returns NULL for every field of a NULL Blob v2
value, so `payload.size = 0` matches only empty blobs and `payload.size
IS NULL` matches NULL blobs.

DataFusion 54.1.0 extracts struct children without combining their
validity with the parent struct. Lance descriptors keep nullness in the
parent and can store zero-filled children beneath NULL blobs. The
adapter derives the descriptor schema from Lance's default projection,
marks descriptor children nullable, and masks them before SQL
evaluation. Schema derivation requires no query request, and the mask
handles struct and list nesting while preserving existing child nulls
and value buffers without fetching blob bytes.

Filter classification follows the referenced struct field path. Blob
predicates stay above the mask; scalar siblings such as `extra.note`
retain exact filter pushdown even when `extra.clip` is a blob.

**Merge hold:** as requested in the [review
discussion](https://github.com/lancedb/lancedb/pull/4571#issuecomment-6253443077),
blob SQL INSERT compatibility remains pending a separate write-mode
design. DataFusion uses the provider's single schema for reads and
INSERT coercion, so the descriptor schema still rejects blob write
structs. This follow-up does not restore those INSERTs; the merge should
wait until the write contract is settled.

Validation:

- Reproduced the original NULL blob returning `kind = 0` before enabling
the mask.
- Reproduced both the extra remote query and lost nested sibling
pushdown before the follow-up fixes.
- All 441 DataFusion adapter tests passed after merging main at
`e6789a41`, including SQL projections, filters, mixed predicates, LIMIT,
COUNT, schema agreement, sliced arrays, nested structs, list variants,
and EXPLAIN assertions for scalar sibling pushdown. The blob fixtures
use the current four-field write format, and NULL-blob filtering is also
covered with row IDs and row addresses enabled.
- Remote constructor regression passed: one describe request and no
query request, with nullable descriptor fields preserved.
- `cargo fmt --all`
- `cargo clippy --locked --profile ci -p lancedb --features remote
--tests --examples -- -D warnings`

Fixes #4569

<!-- lance-gatekeeper-fix:v1 agent=2ca7389efd48d11a8debe9111abeb00e
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-10-27 14:11:39 +08:00
Lance Release 37b46f6b4f Bump version: 0.41.0-beta.14 → 0.41.0-beta.15 2026-10-27 01:43:00 +00:00
Wyatt Alt fe07c25a0a fix: refuse Function-bound writes by column instead of by table (#4686) 
A table with a Function binding rejected every update and merge_insert,
whatever columns the write named: updating an unrelated column, or a
merge that only inserts rows, failed with "not supported on a table with
registered Function bindings". Expression computed columns on the same
table accept those writes.

Both paths now use the per-column guard that schema evolution already
uses. An update is checked on its SET columns. A merge is checked on the
columns a matched row would take from the source, every source column
but the join keys; an insert-only merge is an append, and its new rows
get their Function outputs from the next refresh. Writes to a binding's
inputs or outputs are still refused, now naming the column.
 2026-10-26 18:38:31 -07:00
lancedb-gatefixer[bot]andXuanwo e3dce2cbad fix(rust): prevent oversized SQ remap panics (#3861) 
## Summary

- preflight IVF-SQ and IVF-HNSW-SQ storage segments touched by
compaction
- preserve unknown legacy fragment coverage as conservatively affected
- cover both fixed-size vector and multivector SQ storage dimensions
- reject only affected partitions whose fixed-size-list child exceeds
Arrow's `u32` take-index limit
- return an actionable `InvalidInput` error before Arrow can panic

## Root cause

Arrow 58 builds fixed-size-list child indices with `u32`. For a large SQ
partition, `partition_rows * vector_dimension` can exceed `u32::MAX`;
the child range then wraps inside `take_fixed_size_list`, producing the
reported `ArrayData::slice` assertion during index remapping.

The preflight now reads raw segment metadata, mirrors Lance's
fragment-bitmap selection (including `None` as affected/unknown), opens
only affected vector segments, and checks their actual storage quantizer
and partition sizes. It derives the stored dimension for both
`FixedSizeList` vectors and flattened `List<FixedSizeList>`
multivectors. Stable-row-ID datasets, deferred remapping, custom
remappers, unaffected segments, non-SQ indexes, safe partitions, and
no-op compactions are unchanged.

Because the upstream Arrow kernel still has the limit, an affected index
must be recreated with default IVF partitioning (or enough partitions to
stay below the reported maximum) before compaction. This changes the
failure from a process panic to a targeted recovery error; it does not
silently alter index structure.

## Validation

- `cargo fmt --all -- --check`
- `cargo test --quiet --features aws,remote -p lancedb
table::optimize::tests --lib`
- `cargo test --quiet --features remote -p lancedb
table::create_index::tests::test_create_index_ivf_hnsw_sq`
- `cargo check --quiet --features remote --tests --examples`
- `cargo clippy --quiet --features remote --tests --examples -- -A
clippy::redundant-pub-crate` (the allowance is for pre-existing Rust
1.97 warnings in remote-table code)

Fixes #2866

<!-- lance-gatekeeper-fix:v1 agent=ba2f07b1ee6f9e68a0de1961c164f70e
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 08:27:27 +08:00
lancedb-gatefixer[bot]andXuanwo a9fb16ac44 fix: return cosine-scaled ANN distances (#3852) 
## Summary

- convert unrefined cosine ANN scores from normalized squared-L2 scale
to the public cosine scale
- translate public distance-range bounds to the internal ANN scale
before execution
- add regression coverage for indexed distances and range filtering
while leaving refined, exact, and flat paths unchanged

## Root cause

Cosine PQ, SQ, and RQ quantizers normalize vectors and search with
squared L2 internally. The unrefined native query plan returned those
internal scores directly, making the values twice the public cosine
distance for unit-normalized embeddings.

## Validation

- `cargo test --quiet --features remote --tests`
- `cargo check --quiet --features remote --tests --examples`
- `cargo clippy --quiet --features remote --tests --examples`
- `cargo fmt --all -- --check`
- rebuilt the Python extension and reran the issue reproduction through
the Python API

Fixes #3530

<!-- lance-gatekeeper-fix:v1 agent=fa75cad2d924c92a97215fcab4e21b14
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 06:40:06 +08:00
lancedb-gatefixer[bot]andXuanwo 3352bf49df fix(python): expose optimize compaction options (#3870) 
## Summary
- expose typed compaction options on synchronous and asynchronous table
optimization
- translate validated Python options into native Lance compaction
settings
- document the row-based steady-state limit and first-pass/source-budget
caveats
- carry the Lance beta.15 fragment-exclusion option through the typed
Python API

## Root cause
The native Python optimize binding always constructed default compaction
settings, and the public AsyncTable API offered no way to lower the
fragment row target. Wide-row fragments therefore remained compaction
candidates on every later optimize call, repeatedly rewriting the
accumulated table.

## Limitations
This is an explicit row-based mitigation, not byte-native task sizing or
a hard memory cap. Callers must derive target_rows_per_fragment from
expected row size. Existing oversized fragments are rewritten whole on
the first pass; max_bytes_per_file limits output file size rather than
compaction memory; and source budgets admit whole planned tasks, so a
budget below the first task performs no compaction. Splitting tasks by a
byte budget or returning a planner error requires a Lance-side change.

## Validation
- cargo check --release -p lancedb-python
- cargo clippy --quiet -p lancedb-python
- cargo fmt --all --check
- ruff format --check python tests
- ruff check python tests
- uv run --no-sync pytest python/tests/test_table.py -k
optimize_compaction -q (16 passed)

Fixes #2325

<!-- lance-gatekeeper-fix:v1 agent=9b58eee0c79040b19540e259af0e676c
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-27 03:59:52 +08:00
Xuanwo 311ded06d0 fix(ci): acknowledge Arrow-pinned zstd advisories (#4683) 
RustSec published four `zstd` advisories after #3819 merged, causing the
`deny` job to fail on `main`. `arrow-ipc 58.x` pins `zstd ^0.13`, while
the patched releases require `zstd >= 0.14.3`; moving to Arrow 59+
requires a coordinated DataFusion and Lance upgrade.

This records temporary exceptions with their reachability rationale.
LanceDB uses the `zstd::bulk` API through `arrow-ipc`; the affected
`with_context`, seekable, and streaming-decoder APIs are not used, and
the `zstd-sys` advisory is wasm32-only while wasm32 is not a shipped
target.

Validation: `cargo-deny 0.20.2` reports `advisories ok, bans ok,
licenses ok, sources ok`.
 2026-10-27 01:02:28 +08:00
Drew 436c209580 docs: document Blob V2 in the multimodal guide (#4623) 2026-10-26 09:37:28 -07:00
lancedb-gatefixer[bot]andXuanwo 7952ee2a71 fix(python): cover FP16 GPU index queries (#3819) 
Keep FP16 vectors through IVF-PQ index creation and search in the
synchronous and asynchronous quickstarts. Add native and conditional
CUDA regression coverage using a float64 query against an explicitly
typed FP16 column. This is test and documentation-example coverage for
dtype preservation already present in Lance; it does not change
production code.

Related #2120.

Validation on `bbad20940632b21a9ee85bdcb46b4cc19dbd65e3` with Python
3.12.12, Lance 15.0.0-beta.1, PyArrow 24.0.0, and Torch 2.14.1:

- The native FP16 parameter, Arrow FP16 ingestion test, and both
quickstarts passed: 4 passed, 1 CUDA-only skip.
- The two affected test files passed: 386 passed, 1 CUDA-only skip (`-m
"not slow and not s3_test"`).
- Ruff 0.9.9 lint and formatting checks passed for the Python package.

Real CUDA execution has not been performed. Torch is installed, but
`torch.cuda.is_available()` is false; `test_create_f16_table[cuda]`
skips with `CUDA not available`. The native CPU result does not
establish that the original GPU failure is resolved. Issue #2120 remains
open for verification on an actual CUDA device.

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-26 23:31:19 +08:00
KarthikSandXuanwo 180eaaf68e fix!: reject unsupported remote connection and table options (#4486) 
Fixes #4399.

Remote connections currently discard local-only settings, including
table write options that determine storage format. The synchronous
Python client also binds the fourth positional `create_table` argument
to `on_bad_vectors`, so an intended overwrite is sent as a create.

This change aligns the remote Python signature with
`DBConnection.create_table`, preserves the legacy remote timeout
keywords, and raises `ValueError` for other unknown connection keywords.
The shared Rust client rejects unsupported connection, create, and open
options before sending HTTP requests; open validation also runs before a
table-cache hit. Python reports these as `NotImplementedError`,
including explicit `enable_v2_manifest_paths=False`. Supported
Enterprise connection-level storage options remain available.

This implements option 2 from the issue: explicit rejection. It
deliberately does not add unverified server write parameters. Calls that
previously supplied ignored options must omit them to use server-managed
storage and caching. Database URI prefixes are already preserved on
current main and are unchanged here.

Regression tests cover Python sync/async, TypeScript create/open calls,
and the Rust remote client, including empty-table creation, overwrite
mode, and cached table handles. Python API documentation describes the
behavior and corrected positional order.

Reconciled with main at `0ea932da`, preserving upstream blob and
error-handling tests and updating the Python test that previously
expected open options to be ignored.

Validation on the merged source:

- Rebuilt the Python native extension; 304 tests passed across remote
options, database, remote database, and session suites.
- Rust remote database suite: 135 passed. Rebuilt the Node native
extension; TypeScript remote suite: 62 passed. TypeScript compilation
and documentation generation passed.
- Full Python SDK lint and formatting pass (159 files); full TypeScript
lint and formatting pass (66 files). Rust formatting and `git diff
--check` pass.
- Repository-wide Python lint/format reports existing issues in
unchanged `docs/web-tests` files (49 lint findings and 11 formatting
files).

Environment: Ubuntu under WSL2, Linux x86-64, CPython 3.12.13, Rust
1.97.0, and Node.js 22.16.0. Locked dependencies and the repository `ci`
build profile. Remote tests use HTTP fixtures; this does not claim
validation against the original ARM64 single-node deployment.

Documentation review follow-up: added a draft Enterprise migration note
in `docs/web/basics/sdk.mdx`. It explains that `db://` connections
reject local session/cache and per-table storage/write options, while
supported connection-level Enterprise storage options remain available.
Following the repository's release conventions, the note stays in an MDX
comment below the migration list until the behavior is released. Vale
3.24.0 with the pinned LanceDB style package reports no findings; `git
diff --check` passes. This follow-up changes documentation only.

BREAKING CHANGE: Remote connections now reject unsupported local
session/cache and per-table storage/write options instead of silently
ignoring them. Callers must omit these options and use server-managed
storage and caching; supported Enterprise connection-level storage
options remain available.

Integration with current main preserves its synchronous API signatures,
including the remote `create_table(mode=None)` default, and the table
compatibility defaults. The positional-order repair is already on main
via #4608; this PR retains the distinct unsupported-option rejection
behavior.

CI follow-up: the inherited IVF search-effort test occasionally measured
recall 0.375 against its unchanged 0.5 threshold after random index
training. A matching local reproducer observed this once in 30 builds.
The fixture now places the eight exact neighbors together, preserving
the 16-partition setup, two writes, all probe/precedence assertions, and
the recall threshold. The adjusted fixture returned recall 1.0 in 100
builds; the full query and remote suites passed (102 tests). This
stabilizes test data and does not change search behavior or promise
production recall.

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-26 21:31:13 +08:00
KarthikSandXuanwo 982b2a1229 fix(python): restore sync remote connection and table API parity (#4473) 
Fixes #4400.

Sync remote connections fail on `uri`, `len(db)`, and `name in db`, and
remote tables lack `namespace`, `id`, and `replace_field_metadata`.
Share the paginated connection helpers and table identifier contract
through the base classes, and delegate remote metadata replacement to
the existing async implementation. Unsupported remote session and
read-consistency properties raise explicit `NotImplementedError`s.

Reconciled with main at `3dd57978`, including the synchronous method
signatures from #4498. The local session fix has already landed in
#4433, so this PR preserves its supplied-session-or-None contract and
adds no Rust/PyO3 session access. Namespace connections retain their
writable properties.

Existing custom `Table` subclasses remain instantiable: `namespace`
defaults to a fresh empty list, and deprecated `replace_field_metadata`
delegates to `update_field_metadata` with `replace=True` while retaining
its `None` return. Native and remote overrides are preserved. Regression
tests instantiate an old-style subclass through normal ABC class
creation and verify identifiers, independent namespace lists,
deprecation, and exact metadata delegation.

Historical validation at `03c43f7b` before the latest main integration
and subclass compatibility repair, Ubuntu x86-64, Python 3.12.13:

- Rebuilt the native extension with the repository `ci` profile and
locked Python dependencies.
- 273 tests passed across `test_sync_api_parity.py`, `test_db.py`,
`test_remote_db.py`, and `test_session.py`.
- 84 namespace and catalog compatibility tests passed (357 tests total).
- Python SDK lint and formatting pass (158 files); changed-file checks
and `git diff --check` pass.
- Repository-wide lint/format checks report existing issues in unchanged
`docs/web-tests` files (49 lint findings and 11 formatting files). These
are outside this patch.

Remote tests use the existing HTTP fixtures through the rebuilt native
client; the original aarch64 server deployment was not recreated.

Validation after merging main `3dd57978` and the compatibility repair
(macOS ARM64, uv-managed Python 3.12.12, rebuilt editable native
extension):
- 60 focused parity, signature, and error-contract tests passed.
- 381 database, remote, session, namespace, catalog, and sync parity
tests passed; 2 platform-dependent tests skipped.
- The two new subclass regressions fail against the pre-repair
implementation with the expected abstract-member instantiation error,
then pass with the repair.
- Python SDK Ruff lint/format passes (167 files); repository-wide
findings remain confined to unchanged docs/web-tests (49 lint findings,
11 formatting files).

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-26 20:12:53 +08:00
KarthikSandXuanwo 3dd579785e fix(python)!: align synchronous table method signatures (#4498) 
Closes #4396.

Calls written against `Table` can fail or bind different arguments when
used with `RemoteTable`. Align the affected sync signatures, positional
order, and defaults; preserve renamed remote keywords as deprecated
aliases. Share legacy vector configuration and tokenizer inference,
forward index tuning and timeout options, and accept `fast_search`
consistently.

Preserve current upstream conversion behavior: local
`to_polars(batch_size=...)` returns a LazyFrame; remote tables accept
the same argument signature but raise the documented
unsupported-operation error and direct callers to
`table.search().to_polars()`. The abstract `Table` method retains its
unsupported-operation error, with the implementation confined to
`LanceTable`.

Regression coverage checks argument binding, deprecated aliases, option
forwarding, native behavior through both wrappers, HTTP index requests,
and remote conversion rejection. The merge preserves current tokenizer
defaults, remote query defaults, HNSW-PQ support, and
unsupported-operation handling.

Runtime validation at `c4fb01f2` (Linux x86_64, uv-managed Python
3.12.15):
- Focused Ruff lint and formatting passed for all four changed Python
files.
- Rebuilt and installed the current native extension successfully with
the repository's `ci` Cargo profile.
- Seven regression suites passed: `test_errors.py`,
`test_table_signatures.py`, `test_remote_db.py`, `test_table.py`,
`test_index.py`, `test_fts.py`, and `test_query.py`: **874 passed, 1
deselected**, with slow and S3 tests excluded.
- CI caught an abstract-base conversion regression in the initial merge.
The existing error-contract test reproduced it locally (1 failed, 7
passed); restoring the implementation to `LanceTable` fixes it while
preserving signature parity.
- Full-root Ruff reports 49 lint errors and 11 files needing formatting,
all in unchanged upstream `docs/web-tests` files. The four changed
Python files pass both checks.
- Validated against upstream `197403e8`, using Lance 15.0.0-beta.1 and
LanceDB 0.41.0-beta.14 bindings.

Historical validation before this merge: 618 tests passed and one was
deselected. That result does not establish correctness of the current
merge. The original client-side reproduction produced nine
argument-binding errors without contacting a server; the initial
regression suite failed 32 cases before the fix and passed 39 after it.

Documentation review follow-up: restored the `batch_size` parameter
description on the abstract `Table.to_polars` method so it appears in
the public API reference. This adds six docstring lines and changes no
runtime behavior. After this edit, all 47 focused error-contract and
signature tests pass; changed-file Ruff lint/format and `git diff
--check` pass. Repository-wide Ruff still reports the same 49 findings
and 11 formatting files in unchanged `docs/web-tests`. The full
documentation site and seven-suite runtime run were not repeated for
this documentation-only follow-up.

The reported ARM64 production-server deployment has not been recreated.
HTTP integration tests use the repository's local mock server.

BREAKING CHANGE: Remote synchronous table methods now use the positional
argument order and defaults documented by `Table`. Migrate calls using
the former remote-only positional order to named arguments, particularly
`create_index` and `search`; pass `index_type` by keyword to
`create_scalar_index`. Remote scalar and vector index creation now
default to `replace=True`; specify `replace=False` to retain rejection
of an existing index. Renamed remote keywords remain supported as
deprecated aliases, as documented above.

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-26 19:28:44 +08:00
lancedb-gatefixer[bot]andXuanwo 66357275bb fix: inherit Lance defaults for IVF RQ quantization (#4585) 
`IvfRq()` forced `num_bits=1` through Python's Rust binding, overriding
Lance's five-bit default. The shared native index creation path also
substituted one bit when the Rust or Node builder left the value unset.

Make Python's bit count optional and forward only explicit values. Use
`RQBuildParams::default()` for an unset native bit count, and omit unset
values when dispatching to pylance for accelerator builds. Update the
Python, Rust, Node, and vector-index documentation.

New indexes with an unset bit count now inherit Lance's default,
currently five bits, across Python, Rust, and Node. Five-bit vector
codes use roughly five times the storage of one-bit codes, plus other
index overhead. This also changes reader compatibility: indexes created
with `num_bits >= 2` use a newer on-disk layout that some older LanceDB
versions cannot read. Upgrade readers to versions that support multi-bit
IVF_RQ, or explicitly set `num_bits=1` (for example, `IvfRq(num_bits=1)`
in Python) when sharing indexes with readers limited to one-bit IVF_RQ.

Validation:

- Reproduced six regression failures before the fix: one-bit default
metadata, rejected `None`, and incorrect remote/accelerator forwarding.
- After merging `main` through `b762ec22`, rebuilt both native bindings
against Lance 15.0.0-beta.1. Python validation passed with 106 tests and
15 optional Ollama skips. Coverage includes the full index suite,
omitted and explicit bit counts, reading Lance-created IVF_RQ indexes,
accelerator forwarding, CUDA-unavailability handling, timeout
validation, remote requests, and table/namespace listing behavior.
- Rust formatting and Python package Ruff lint/format checks pass. Node
`pnpm lint`, `pnpm build` (including TypeScript checks), `pnpm run
docs`, the full connection suite, and the existing IVF_RQ regression
pass. Generated documentation adds no further changes.

The merge preserves upstream's explicit RaBitQ heading anchor alongside
the `IvfRq()` example and five-bit default/reader-compatibility
guidance.

Repository-wide `ruff check .` still reports 49 existing errors in
unchanged documentation examples under `docs/web-tests`; root formatting
also finds 11 unchanged documentation example files. The Python package
checks pass.

Fixes #4582

<!-- lance-gatekeeper-fix:v1 agent=6ef9406f0142b1ab66d53d9d56098223
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-26 18:57:25 +08:00
devjoinedthechatdevjoinedthechatXuanwo
 		22fc2a2087 fix(python)!: put OpenCLIP models in eval mode before embedding (#4536) 
`OpenClipEmbeddings` loads its model with
`open_clip.create_model_and_transforms()`, which hands the model back in
training mode (the open_clip README calls `model.eval()` right after it
for that reason). `OpenClipEmbeddings` never switches it out of training
mode, so layers that behave differently during training stay active
while we embed: stochastic depth in the timm-based image towers
(ViTamin, ConvNeXt, MobileCLIP2-L-14), dropout in the Hugging Face text
towers (`xlm-roberta-*`, `roberta-ViT-B-32`, `nllb-clip-*`), and batch
norm in the ResNet towers loaded with non-OpenAI weights. The
`torch.no_grad()` blocks turn off autograd but don't change how these
layers behave.

The default `ViT-B-32` has none of those layers, which is probably why
nobody has run into this. With dropout or stochastic depth active, the
same input gives a different vector on every call, so the vector stored
at ingestion and the one computed for an identical query don't match.
With batch norm in training mode, an image's vector depends on the other
images in its batch:

```python
import numpy as np
from PIL import Image
from lancedb.embeddings import get_registry

func = get_registry().get("open-clip").create(name="ViTamin-S", pretrained="datacomp1b")
image = Image.new("RGB", (256, 256), "red")
a = func.generate_image_embedding(image)
b = func.generate_image_embedding(image)
print(func._model.training, np.array_equal(a, b))
# before: True False
# after:  False True
```

The fix calls `model.eval()` once after loading, same as the SigLIP,
ColPali and ImageBind embedding functions already do.

The new test swaps in a stand-in for the `open_clip` module, so it runs
without torch or a model download, and checks that the model handed back
by `create_model_and_transforms` ends up in eval mode.

BREAKING CHANGE: Embeddings change for OpenCLIP models with
training-sensitive layers, including ViTamin, ConvNeXt, MobileCLIP2,
Hugging Face RoBERTa/XLM-RoBERTa/NLLB text towers, and ResNet models
loaded with non-OpenAI weights. For tables embedded with affected
models, regenerate the stored embeddings in eval mode and rebuild their
vector indexes before querying with the new embeddings. Rebuilding an
index alone does not regenerate embeddings. The default ViT-B-32 model
is unaffected.

Co-authored-by: devjoinedthechat <235875317+devjoinedthechat@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-26 18:17:14 +08:00
lancedb-gatefixer[bot]andXuanwo 7ed03e1c40 fix(python): honor bad-vector policies for pandas NaNs (#4503) 
Pandas converts NaN vector components to Arrow nulls. Both Python's NaN
mask and Rust's default append validator ignored null children, so
`add()` silently stored incomplete vectors despite
`on_bad_vectors="error"`.

Reject null components in the existing Rust ingestion validator, and
include null components in Python's bad-vector detection and filling.
`drop` removes affected rows, `fill` repairs missing components while
preserving valid ones, and `null` replaces affected vectors. Explicit
null components in Arrow and list inputs follow the same policies. Rust
allows whole null vectors and retains the explicit
`NaNVectorBehavior::Keep` opt-out.

The ingestion guide and synchronous/asynchronous `create_table()` option
docs describe null components, all four policies, and filling that
preserves valid components.

Validation:

- Reproduced the default pandas append failure with the regression test
before changing the Rust validator.
- Rebuilt the native Python extension from the merged sources. The
Python table and utility suites passed: 481 passed, 1 skipped. Coverage
includes create/append/overwrite, pandas/rows/Arrow inputs, every
bad-vector policy, float16/32/64, sliced and chunked arrays, all-null
components, and hidden children of null lists.
- All 43 focused Rust tests passed: 7 validator tests and 36 ingestion
tests.
- Rust Clippy with `remote`, tests, and examples passed. Ruff lint
passes for the Python package, formatting checks pass for modified
Python files, and `cargo fmt --all` is clean.

- Documentation follow-up: all 56 focused vector-policy tests passed;
Python package lint and creation-docstring formatting checks passed.

Repository-wide Ruff reports 49 existing errors in untouched
documentation examples. Create/overwrite retain their existing
`RuntimeError` wrapping for errors raised while consuming lazy Arrow
readers.

Fixes #4435

<!-- lance-gatekeeper-fix:v1 agent=b5108e27d28ef613a78c1e4354c03c48
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-26 17:46:35 +08:00
Joaquin Hui GomezandXuanwo 256db55a51 fix(python)!: align RemoteDBConnection.create_table parameter order (#4608) 
`RemoteDBConnection.create_table` orders its parameters `(name, data,
schema, on_bad_vectors, fill_value, mode, exist_ok, ...)`, while
`DBConnection`, `LanceDBConnection` and `LanceNamespaceDBConnection` all
use `(name, data, schema, mode, exist_ok, on_bad_vectors, fill_value,
...)`. Since `RemoteDBConnection` subclasses `DBConnection`, identical
positional calls bind to different parameters: switching a working
script from `lancedb.connect(path)` to `lancedb.connect("db://...")`
silently turns `mode="overwrite"` into `on_bad_vectors="overwrite"` and
`exist_ok=True` into `fill_value=True`, and nothing client-side
validates either. `inspect.signature(...).bind(None, "t", data, None,
"overwrite", True)` shows it deterministically, no cloud needed.

This reorders the remote signature to match the other three; every
parameter keeps its own existing default (including the remote's
`mode=None`, which its body relies on), and the body already forwards by
keyword so nothing else changes. The new test asserts positional parity
with `DBConnection` through `embedding_functions` and fails on `main`.

Blast radius, stated plainly: only remote callers passing four or more
positional arguments are affected — positions 4–7 move; positions 1–3
and 8 are unchanged and keyword callers are unaffected. No in-repo
caller passes more than two positionals. Anyone relying on the current
order was relying on behaviour that contradicts the documented base
contract and diverges from local connections. Not added here: the
`storage_options`/`data_storage_version`/`enable_v2_manifest_paths`
parameters the remote signature still lacks.

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
 2026-10-26 17:02:36 +08:00
lancedb-gatefixer[bot] e6789a4118 fix: support system columns in DataFusion table adapters (#4667) 
DataFusion plans SQL against the adapter's declared schema, which
omitted `_rowid` and `_rowaddr` even though Lance can project both. SQL
therefore rejected these columns before reaching the scan.

Add `BaseTableAdapter::with_system_columns` to opt into either column
using Lance's nullable `UInt64` fields. Scans without a projection now
select the full declared column list so their output matches the
provider schema. Default adapters retain data-only `SELECT *` and remain
compatible with `INSERT INTO ... SELECT *`; opted-in adapters include
the requested system columns in `SELECT *`.

Regression tests compare SQL results with native queries and cover
projection order, system-only projections, row-ID predicates, scans
without a projection, duplicate/unsupported names, and default SQL
inserts.

Validation:

- DataFusion unit suite: 436 tests passed with `cargo test --locked
--quiet --profile ci -p lancedb --features remote --lib
table::datafusion`.
- `cargo clippy --locked --quiet --profile ci -p lancedb --features
remote --tests --examples -- -D warnings` passed.
- The new builder's documentation example passed its doctest.
- `cargo fmt --all -- --check` and `git diff --check` passed.

Fixes #4666

<!-- lance-gatekeeper-fix:v1 agent=58d267206580227b3820df65af15b0a7
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
 2026-10-26 15:55:37 +08:00
Jack YeandLance Release 197403e891 fix: honor downstream Lance dependency overrides (#4671) 
Consumers that provide a local Lance checkout cannot resolve the
dependency graph because the SDK directly selects a vendored
`lance-arrow`, producing two path packages with the same name and
version. The Rust core now declares its upstream Git dependency
separately so the consuming workspace can override that source
consistently. Python continues inheriting the vendored workspace
dependency, keeping the sliced-struct validity fix in Maturin source
distributions. Standalone SDK builds still select that same fix through
the existing root patch.

Adds a downstream consumer CI regression that resolves a local Lance
override and checks that exactly one canonical `lance-arrow` package is
selected. The Lance version updater also updates the core declaration in
stable, preview and local modes. Public APIs and query behavior are
unchanged.

The workspace declaration lint excludes path dependencies because the
Python binding intentionally uses the bundled source. Duplicate and
unused external declarations remain errors; advisory, license,
banned-package and source checks continue covering the full dependency
graph.

---------

Co-authored-by: Lance Release <lance-dev@lancedb.com>
 2026-10-25 07:19:55 -07:00