mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-30 23:36:52 +00:00
ddb318567a11211a9b75aca6dcf2902ff615f7d8
3358
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jack Ye
|
ddb318567a |
docs: complete SQL reference, enrich Functions drafts, rename OSS to Embedded (#4732)
## Summary - Complete the SQL reference: `ATTACH DATABASE`, the `CHANGES` clause, and `dedupe()`/`duplicate_pairs()` (verified against current sophon source; these were recently renamed from `vector_dedup`/`vector_duplicate_pairs`). Fix the blob-column default-projection claim and add the previously undocumented `blob_descriptor()`. - Enrich the gated Functions draft (still `.mintignore`'d, pre-release): chained Functions over blob outputs, GPU-produced blobs, Secrets, multi-input/output binding, `code=[module]` packaging. - Upgrade the materialized-view and computed-column auto-refresh drafts with precise, verified catch-up semantics, and add the missing `ALTER MATERIALIZED VIEW` / `CREATE ... WITH(...)` sections (also gated — the automatic-refresh scheduler is still dev/staging-only). - Thread one running example (video frames → GPU extract/caption → dedup → auto-refreshing materialized view) through Functions, materialized views, and the LeRobotDataset integration guide instead of a separate tutorial. - Rename "LanceDB OSS"/"OSS" to "LanceDB Embedded"/"Embedded" throughout (52 files); activate the existing availability-comparison mechanism on `basics/offerings.mdx` for a full feature-by-feature comparison table. |
||
 Divyanshu Singh
|
005a9a353a |
ci(java): run Java 11 and 17 builds in parallel with matrix strategy
Split the Java 17 workflow into a matrix for Java 11 and Java 17. Run Checkstyle once in a dedicated Java 17 job and skip the lifecycle-bound Checkstyle execution in the matrix builds to avoid redundant work. Closes #1331. |
||
 Taylor
|
a065b4b9c6 |
feat(python): add NVIDIA Nemotron embedding provider
Adds a Python `nvidia` embedding provider for Nemotron 3.5 Embed 1B and 8B. It works with NVIDIA's hosted API and self-hosted NIM `/v1` endpoints, and uses the existing `openai` client. The OpenAI provider can't send NVIDIA's passage/query input types or pick Nemotron's dimensions. - Documents are embedded as `passage`, searches as `query`. - Default dimensions are 2048 (1B) and 4096 (8B), and `dim` is checked against each model's supported range. Self-hosted model IDs need an explicit `dim`. - Response count, order, dimensions and values are checked. Null and empty text get a null vector without a request. - Each request is retried on its own with LanceDB's retry policy. Invalid input or responses fail immediately. - `NVIDIA_API_KEY` is only sent to the hosted API. Custom endpoints need an explicit `api_key` (a `$var:` reference), and endpoints restored from table metadata must be allowed with `allow_endpoint`. - Request dimensions are sent through `extra_body`, preserving the NVIDIA wire contract while supporting the declared minimum `openai==1.6.1`. Python and text only. Docs are in `integrations/embedding/nvidia`. Originally authored by Taylor (@tsmithv11). ### Testing - `test_nvidia_embeddings.py`: 45 passed, 2 deselected with both the locked `openai==2.44.0` and minimum `openai==1.6.1`. These use the real OpenAI client with mock HTTP transport and real local tables. - Embedding, registry, field-metadata and trust suites: 163 passed, 4 skipped, 10 deselected. - Historical live experiments: hosted 8B passed a five-request smoke test; hosted 1B returned HTTP 500. No live request was made in this stage, so current hosted 1B availability was not verified. Enterprise coverage uses the repository mock HTTP server, not a live deployment. |
||
 BHAWESHBHASKAR
|
891774343e |
feat(python): add CombinedFieldsQuery for BM25F full text search
Add a typed Python `CombinedFieldsQuery` for BM25F search across multiple columns. The query mirrors Lance's combined-fields semantics with ordered per-column boosts and AND/OR term matching. Convert it in both directions at the PyO3 boundary, including nested Boolean queries, and document the Python typed convenience entry. Closes #4652. |
||
 Yang Cen
|
06b6625a94 |
docs: validate release branches before Lance stable approval (#4711)
Freeze the SDK product baseline on a release branch before validating a Lance RC, so later main commits do not change the candidate. Require the RC pin PR's CI and applicable reviews to complete with no blocking findings before Lance stable publication. Draft-skipped reviews are missing evidence, and green CI does not clear a request-changes recommendation. After the upstream release, update only the validated dependency references, rerun checks and refresh review, then publish from the verified release branch. Document Cargo override boundaries, exact commit/artifact evidence, and the workflow's actual stable input (`type=stable`). Validation: Markdown fences, public-scope and diff checks passed; the documented workflow inputs were checked against the repository workflow. |
||
 Jay Pokale
|
280ca41061 |
feat(python): support time and timedelta values in table.update
Support `time` and `timedelta` values in `table.update` by generating explicit Arrow casts. - Preserve Arrow duration/time units and pandas nanosecond precision. - Reject timezone-aware `time` values before data can be changed. - Cover time64 us/ns, duration s/us/ns, negative timedeltas, filters, and nanosecond round trips. |
||
 Wyatt Alt
|
41c1a2a8b1 |
feat(python): declare Debian system packages on a Function
Adds `apt` to `@udf` and `system_packages` to the Rust `PythonEnvironmentSpec`. The server adds the packages to the Function's image before installing its Python environment, for shared libraries that pip and conda cannot provide, such as FFmpeg's for torchcodec. The field is omitted when empty, so existing Function identities are unchanged. |
||
 Ishita Singh
|
079711d4ad |
feat(python): list registered embedding functions
Closes #1605. Adds `EmbeddingFunctionRegistry.list_functions()` to expose registered lookup names in registration order without accessing `_functions`. The method returns an independent snapshot, and every returned name can be passed to `get()`. Tests cover empty registries, aliases and class names, registration order, snapshot isolation, and reset. |
||
 geoffgscott
|
7ea10fa1e0 |
feat(node): expose targetPartitionSize on IVF index options (#4594)
The Node SDK's vector index options only take `numPartitions`. The Rust index builders (`target_partition_size` in `rust/lancedb/src/index/vector.rs`) and the Python SDK already support a target partition size, so this adds `targetPartitionSize` to `IvfPqOptions`, `IvfRqOptions`, `IvfFlatOptions`, `HnswPqOptions` and `HnswSqOptions` and passes it to their builders. When both are set, `numPartitions` is used, as it is for Rust and Python callers. The reason to want it: an index built from a partition count records no target partition size, so `table.optimize()` splits and joins it against the index type's default and can merge small partitions down to one or two. Building with a target records it in the index details, and optimize keeps to it. https://github.com/lance-format/lance/pull/10154 fixes this on the Lance side for indexes built from a partition count; this lets Node users set the size directly. The new tests build IvfPq, IvfFlat, IvfRq, IvfHnswPq and IvfHnswSq indexes with `targetPartitionSize` and check the size recorded in the index details, and check that `numPartitions` wins when both are set. Each option's doc comment states that index type's default target. Docs regenerated with `pnpm run docs`. |
||
 Lei Xu
|
03bd909dab |
test(python): resolve aliased projections in the torch remote mock server
Map dynamic projections by output name to their source columns in the remote torch test mock so permutation take queries preserve the aliased ordering column. Run the regression in the embeddings-installed Python test job and require torch to import so a missing dependency cannot skip coverage. |
||
 max
|
9ffd3d33e1 |
test(python): remove fixed per-test delays from mock HTTP servers
The Python test jobs spend much of their pytest time waiting on the in-process mock HTTP servers. 1. Each server ran `serve_forever()` with the default 0.5 s poll interval, so every `server.shutdown()` blocked for up to 0.5 s. Around 300 tests use such a server (test_remote_db, test_authz/test_catalog, test_listing, the permutation helpers), about 150 s per test job on every platform. A shared `start_server_thread()` helper in `tests/utils.py` serves with a 0.01 s poll interval, so shutdown returns in about 10 ms. 2. The servers bind IPv4 only, but clients used `http://localhost:<port>`. The client tries ::1 first and falls back after the happy-eyeballs delay, about 0.3 s per connection on Windows. The mock handlers close the connection after each response, so one listing test took 9.5 s on Windows CI versus 0.6 s on Linux. Clients now use 127.0.0.1, as test_catalog, test_listing and test_namespace already do. Measured on Windows with the 0.41.0b6 wheel and `-m "not slow and not s3_test"`: full suite 546.7 s to 212.3 s; test_remote_db.py, test_listing.py and test_catalog.py together 284 s to 14 s on current main. Collected test ids are identical (1898) and pass/fail/skip results match. No assertion, test or marker changed. |
||
  lancedb-gatefixer[bot]andXuanwo
|
913ee160b0 |
test(python): cover vector index prewarming by name (#3859)
## Summary - cover vector-index prewarming through the Python API using the logical index name returned by list_indices - prevent regression to interpreting a vector index name as a schema column ## Root cause The Lance version used by the reported release opened a vector index by treating the prewarm argument as a column before resolving the index metadata. Current LanceDB dependencies contain the upstream metadata-based resolution, but LanceDB only exercised prewarming for FTS indexes, leaving the vector behavior unguarded. ## Validation - uv run --extra tests pytest python/tests/test_index.py::test_create_vector_index -q - python/.venv/bin/ruff format --check python/python/tests/test_index.py - python/.venv/bin/ruff check . Fixes #2897 <!-- lance-gatekeeper-fix:v1 agent=c2f8273221061d8e2b6f7393066e68c6 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
 Bilal Atique
|
65954bf9cc |
docs(python): document connection_factory on StreamingDataset
Closes #3693. `columns` is documented on `StreamingDataset` now, but `connection_factory` still wasn't. This adds it to the parameter list, in signature order. The text follows `__getstate__` / `__setstate__`: with a factory, pickling the dataset replaces the table's reopen state with its name, and each DataLoader worker calls the factory to reopen the table. The factory is pickled along with the rest of the dataset, so it has to be picklable, and credentials stay out of the pickle only when the factory loads them in the worker instead of capturing them (the same guidance as `Permutation.with_connection_factory`). Without a factory, the table's own reopen state is pickled. This picks up #3697, which documented the same parameter and was closed by its author without a merge. |
||
|
Wyatt Alt
|
3506b43c03 |
feat(python): accept input_schema or output_schema alone in @udf (#4731)
Binding a nullable column needed an explicit input schema, and that forced spelling out the output schema too. Either schema can now be given alone; the other side is inferred from annotations. The udf docstring now says a bare annotation is non-nullable and `Optional[T]` binds a nullable column, and that a `bytes` input counts against the per-row input limit, with Blob V2 and `BlobReader` for large values. |
||
 medimedi
|
088f4d670e |
docs(rust): use LanceDB Arrow re-exports (#3908)
## Summary - explain that Rust callers should use the Arrow crates re-exported by `lancedb::arrow` - update the crate quick-start doctests to build Arrow values from those re-exports - update the checked-in `simple` example so it cannot drift to an incompatible Arrow crate version Fixes #1207. ## Validation - `cargo fmt --all -- --check` - `cargo metadata --no-deps --format-version 1 --quiet` - static checks that the documented crates are publicly re-exported and both examples use them - `git diff --check` - attempted `cargo check --quiet -p lancedb --example simple`; dependency compilation stopped in the unmodified `lance-encoding` build script because `protoc` is not installed in this environment |
||
|
Wyatt Alt
|
5fbf9fd9f0 |
docs(python): note that packaged Functions import lance_blob.BlobReader (#4730)
Function authors annotate inputs with lancedb.BlobReader, but the packaged source binds that annotation to the worker's lance_blob.BlobReader, so grepping the artifact for the client class finds nothing. Say so on the class. |
||
|
lancedb-gatefixer[bot]
|
04e509278e |
fix(python): clarify schemas for batch UDF annotations (#4727)
`@udf` documented batch annotations without explaining that `pa.Array` and `pa.RecordBatch` do not specify concrete Arrow element types or field schemas. Document the requirement for both explicit schemas (including class `__call__` methods), add an array example, and make inference errors point callers to `input_schema` and `output_schema`. Fixes #4721 Validation: - Five new regression cases pass; all four diagnostic cases fail with the original implementation. - Function packaging, first-class Function slice 1/2 tests, and UDF doctests: 168 passed, 3 failed. This includes the upstream BlobReader annotation coverage. The three existing packaging tests fail while importing temporary modules, before UDF registration; all three failures also reproduce with the unchanged source and tests at `057a39a1`. - Rebuilt the native extension in the uv-managed tests/dev environment. - Python lint and formatting checks pass (`ruff check python`, `ruff format --check python`); `cargo fmt --all` also passes without changes. - Repository-wide `ruff check .` reports 48 existing errors under `docs/web-tests`. <!-- lance-gatekeeper-fix:v1 agent=2c6567ee3f128d3a0c3b9c68e36eca38 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
741cb9ad4e |
fix(python): implement remote namespace and table existence checks (#4725)
Remote Python connections inherited the unsupported-operation stubs for `namespace_exists` and `table_exists`, so both raised `NotImplementedError` even when namespace and table operations worked. Add the missing remote overrides, async methods, and PyO3 bindings using the existing namespace describe and table open paths. Only an explicit not-found result returns `False`; permission, transport, and server errors propagate. This preserves authentication, dynamic headers, retries, and nested namespace handling, and rechecks cached tables before reporting that they exist. The async namespace helper also recognizes native `NamespaceNotFound` errors so missing local namespaces return `False`. The table and namespace guides document Python remote support and distinguish it from the unsupported `exists` routes used by direct REST namespace clients. Validation: - All 40 focused regression cases pass, covering successful and missing lookups, local and remote namespaces, dynamic headers, permission/server errors, invalid table identifiers, and cached tables that disappear. - The original inherited stubs reproduce the reported `NotImplementedError`; the local namespace regression reproduced the native missing-namespace `RuntimeError` before its fix. - Full remote connection suite: 255 passed. Local namespace tests: 14 passed. - `cargo clippy --quiet -p lancedb-python --features remote --lib`, `cargo fmt --all`, Python Ruff lint, and Python formatting checks pass. - Root `ruff check .` reports 48 existing errors in documentation examples under `docs/web-tests`; the Python subtree passes and those unrelated files are unchanged. Fixes #4722 <!-- lance-gatekeeper-fix:v1 agent=4d369a0681756fdb5a8b35853d7f5607 generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
fc5170cbed |
fix(python): expose function source digests (#4729)
Function descriptions include `source_digest`, but both the Rust response model and Python `FunctionVersion` discarded it. Clients therefore could not compare a catalog entry with `udf_def.registration_request.artifact.digest`. Preserve the digest through Rust serialization and expose it on Python `FunctionVersion`, with a documented Rust accessor. Missing or null digests decode as `None`, keeping older responses compatible. Regression tests cover serialization, immutability, registration jobs, get, and synchronous/asynchronous list responses while keeping the image digest independent. Validation: - Confirmed that the original Python model silently discarded a server-provided source digest. - Rebuilt the Python extension in the uv-managed environment; both focused Function test files passed (106 tests). - Focused Rust Function model tests passed (10 tests). - Full workspace `cargo check --quiet --features remote --tests --examples` and `cargo clippy --quiet --features remote --tests --examples` passed. - Rust formatting and Ruff checks/formatting for changed Python files passed. - Repository-wide Ruff reports 138 existing violations in unchanged docs/CI files. Fixes #4723 <!-- lance-gatekeeper-fix:v1 agent=c145f25136f7b908217e04bde8a2548b generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
3035636bed |
fix(python): restore blob extensions in table schemas (#4728)
Reading a Blob v2 table in a fresh process could return its storage struct if neither `blob()` nor pylance had registered the extension. Both local and remote schema reads were affected, so `field.type` could depend on import order. Restore metadata-only blob fields on every table schema read through the existing BlobType resolver and Arrow's schema deserializer. Preserve nested fields, nullability, custom metadata, and both supported storage layouts. The fallback deserializer now honors the stored shape too. Fresh-process regressions cover local and remote tables, sync/async reads, and operation with and without pylance. The four native cases failed before removing the native-table guard. Existing merge-insert tests now require the restored extension type while continuing to verify updated and inserted blob contents. Validation: - All 12 fresh-process local/remote schema regression cases pass. - `uv run --extra tests pytest python/tests/test_table.py python/tests/test_blob.py python/tests/test_namespace.py python/tests/test_util.py -q -m 'not slow and not s3_test'`: 835 passed, 11 skipped. - Python package Ruff lint and formatting checks pass. - Repository-wide `ruff check .` reports 48 pre-existing diagnostics in unchanged `docs/web-tests` files. Fixes #4719 <!-- lance-gatekeeper-fix:v1 agent=67c656bb5889e698efce5f2fd9d58ea7 generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
ca60b402aa |
fix(python): reject invalid punctuation in UDF names (#4724)
Function names containing periods or hyphens passed `@udf` validation and failed only when registered with the server. Restrict the name pattern to ASCII letters, digits, and underscores, preserving the existing first-character rule so invalid names fail when the decorator runs. Document the accepted format and add regression coverage for dotted and hyphenated names, including trailing punctuation, alongside valid name overrides. Fixes #4720 Validation: - The four new invalid-name cases failed before the validator change because no `ValueError` was raised. - `cd python && uv run --extra tests pytest python/tests/test_first_class_function_slice2.py -q`: 86 passed. - Ruff 0.9.9: `ruff check python` and `ruff format --check python` passed. - Repository-root `ruff check .` reports 48 pre-existing errors in `docs/web-tests/`. <!-- lance-gatekeeper-fix:v1 agent=5e2441d3d264b685a6cee3c3c21ab048 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
|
lancedb-gatefixer[bot]
|
5f6f7b591d |
fix(python): bind packaged BlobReader annotations to the worker type (#4726)
Packaged UDFs omit globals used only in postponed annotations, so a `BlobReader` input has a correct wire signature but `typing.get_type_hints()` on the loaded function raises `NameError`. Use the resolved input signature to bind reader annotations to an imported `lance_blob.BlobReader` alias. Preserve signature nullability and avoid collisions with existing globals. Regression tests cover functions and callable classes, eager and postponed annotations, aliases, forward references, qualified names, and body globals. Validation: - Confirmed the new regression fails with `NameError: name 'BlobReader' is not defined` against the original packager at `057a39a1c70e15c78461bb7e3db8cef4c067cd3e`. - The packaging, first-class Function slice 1 and 2, and BlobReader suites pass all 159 tests with Python file-import directory caches refreshed before lookups. The standard run passes 156 tests; three temporary-module import failures reproduce unchanged on the original packager because directory timestamps do not advance between file writes in this environment. All 37 new regression cases pass in the standard run. - Python Ruff lint and formatting checks pass. Repository-wide Ruff lint reports 48 existing errors in documentation examples. Fixes #4718 <!-- lance-gatekeeper-fix:v1 agent=a32bb6efa79217b3c1c5c9d7a7ff9928 generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> |
||
 Nishil Rathod
|
84fc48198b |
docs: add guide for using LanceDB in VS Code extensions
Closes #1849 This was lancedb/docs#371. After lancedb/docs#355 moved the open-source pages, their navigation and the snippet tests into this repository's `docs/web` and `docs/web-tests`, that PR could no longer merge there, so I've moved the same change here. ## Why #1849 asks for an example of LanceDB in a VS Code extension, because people kept running into problems there (#927, #1713). An earlier attempt, #3682, added a README link. @prrao87 asked for something else instead: a quick guide on the docs site, under **Integrations › AI Platforms & Frameworks**, showing how to use LanceDB in an extension and what results look like. The author of #3682 agreed and closed it. This PR is that guide. ## What it adds - **`docs/web/integrations/ai/vscode.mdx`**: a new page, also added to `docs/web/docs.json`. It builds a small extension with two commands. One indexes the workspace's Markdown into a LanceDB table using the built-in `huggingface` embedding function, so no API key is needed. The other searches it by meaning. The page covers the parts that are specific to VS Code: - storing the table under `context.storageUri` instead of a relative path, which is the cause of #927; - `@types/node` ≥ 22 and `skipLibCheck`, which covers #1713 and the DOM-type errors from `apache-arrow`/onnxruntime typings; - pinning `apache-arrow` inside LanceDB's peer range; - marking `@lancedb/lancedb` and `@huggingface/transformers` as esbuild externals, and removing `node_modules/**` from the template's `.vscodeignore`; - storing each file's URI so a hit opens in the right folder of a multi-root workspace; - building one VSIX per platform with `vsce package --target`, with a table mapping each target to LanceDB's native package, and a note that the sample's local model (via `onnxruntime-node`) doesn't run on Alpine; - keeping the Transformers.js model cache in `globalStorageUri`, which only works if it's loaded with `import()`; - a troubleshooting table listing each error message with its cause and fix. - **`docs/web-tests/ts/vscode_search.ts`** and **`vscode_search.test.ts`**: the LanceDB code the page shows (connect, chunk, index, search), plus jest tests for it. - **`docs/web-tests/ts/vscode_extension.ts`**: the extension's `src/extension.ts`. Jest can't run it because it needs the VS Code extension host, so it was tested in a real extension instead (see below). This follows the precedent of `integrations.ts`. - **`docs/web/snippets/vscode_search.mdx`** and **`vscode_extension.mdx`**: generated with the command in CONTRIBUTING.md. ## Changes from lancedb/docs#371 - **New paths only.** The page, tests and snippets are the same content, at their paths in this repository. - **One link.** `/tables/update` doesn't exist under `docs/web`, so the `mergeInsert` link now points to `/ingestion/merge-insert`. - **Heading anchors.** Every heading now has an explicit `{#anchor}`, as CONTRIBUTING.md asks. ## How I tested it - **Snippet tests.** In this repository at `c2c8269f`, `vscode_search.test.ts` passes (2 tests) with the pinned `@lancedb/lancedb` 0.31.0. On Windows the `npm test` script fails before it starts, because it runs `node` on the `.bin/jest` shell shim. I ran `node --experimental-vm-modules node_modules/jest/bin/jest.js --testEnvironment jest-environment-node-single-context vscode_search` instead. - **Lint.** `biome ci` is clean for the three new files, and they carry the SPDX header that `license-header-check.yml` looks for. - **Snippets.** I regenerated them with CONTRIBUTING.md's command, `mdx_snippets_gen.py -s docs/web-tests/py -s docs/web-tests/ts -s docs/web-tests/rs -o docs/web/snippets`, run with `python` since I don't have `uv`. It reproduces the two new snippet files exactly, once the Windows CRLF endings are normalised. As CONTRIBUTING.md warns, it also rewrites `build_with_ai_agents.mdx`, `integrations.mdx`, `search.mdx` and `storage.mdx`. I left those out. - **Links.** `mint broken-links` from `docs/web` reports no entries on the new page, and its three internal links (`/embedding/quickstart`, `/ingestion/merge-insert`, `/search/full-text-search`) resolve to pages here. - **Real extension.** I built a sample extension from exactly the code on the page, scaffolded with `yo code` (TypeScript + esbuild) and using `@lancedb/lancedb` 0.39.0. I ran it with `@vscode/test-cli` in VS Code 1.140 on Windows x64. The integration test runs both commands against a six-file workspace and checks the top hits. The "Sample results" block on the page is copied from that run, and after the review fixes below it reproduces line for line in VS Code 1.141. - **Multi-root workspace.** A second test configuration opens two folders that both contain `docs/auth.md`. It types a query and picks the top hit through the real picker, then checks that the right folder's file opens at the right line. It fails on the code before the review fix and passes after it. - **Packaged VSIX.** I packaged `vsce package --target win32-x64` (200 MB, or 148 MB with the onnxruntime trimming in the Tip). Then I unpacked it outside the project and ran the same tests against the unpacked copy, so they couldn't pick up the project's `node_modules`. They pass. - **Negative checks behind the troubleshooting rows:** - With `node_modules/**` left in `.vscodeignore`, activation fails with `Cannot find module '@lancedb/lancedb'` and the commands report `command '…' not found`. - Without the externals, esbuild fails with `No loader is configured for ".node" files`. - Installing LanceDB into a project that already has `apache-arrow` 21 fails with `ERESOLVE`. - `require("@huggingface/transformers")` returns a different `env` object from `import()`. - **Rendering.** For lancedb/docs#371, `mint dev` rendered the page, its 14 code blocks and the sidebar entry. I haven't re-rendered it from `docs/web`; only the paths, one link and the heading anchors changed. **Not tested:** - macOS. I couldn't reproduce the read-only working directory from #927; the page's fix (an absolute storage path) doesn't depend on the OS. - Linux and macOS VSIX builds. ## Notes for reviewers - I added `_distance` to `.select(...)`, because LanceDB warns that it will stop adding the column automatically. ## AI disclosure I used Claude Code to research the issue and its history, write the sample extension, the tests and this page, move the change here, and draft this description. I ran every check listed above myself, and the outputs quoted on the page come from those runs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
  Will JonesandClaude Opus 5.5
|
4d7a64712f |
docs: adopt Embedded/Distributed terminology across docs and SDKs (#4713)
## Summary Adopts the product terminology guide across the docs site pages, SDK docstrings and error messages. "LanceDB OSS" and local mode become **LanceDB Embedded**. "Remote" as the execution model becomes **LanceDB Distributed**, which LanceDB Enterprise provides. Where free vs paid is the point, pages contrast "LanceDB Embedded (open source)" with "LanceDB Enterprise". This is a first trial of the Embedded name, so reviewers can judge it in context. - **Embedded vs Distributed** for where execution runs (`optimize()`, consistency, branching, maintenance). Main pages define it once: "LanceDB Enterprise runs LanceDB Distributed: …" - **LanceDB Cloud** is kept only for legacy-Cloud specifics (`db://` region, the brute-force KNN limit) and marked legacy. "Cloud and Enterprise" becomes Enterprise or Distributed. - **Plan executor** becomes "executor" or the fleet name. - **Error messages** change, e.g. `optimize is not supported for remote tables.` → `... in LanceDB Distributed.` Tests that match on them are updated. - **Vale rules** (`EmbeddedName`, `OSSProduct`, `RemoteExecution`, `DistributedEnterprise`, `LegacyCloud`, `InternalNames`) enforce the terms in prose. Unchanged: identifiers (`availability.oss`, the `remote` module and feature, `RemoteTable`, `db://`), heading anchors, internal code comments, and Geneva. ## Evidence - Rust lib tests for the changed messages and the Python tests in `test_remote_db.py`, `test_materialized_views.py` and `test_table.py` pass locally. - Not run locally: - The Node tests: `remote.test.ts` and `table.test.ts` have updated assertions. - Vale. - The `experimental_functions`-marked materialized-view tests. - `docs/src` API reference was hand-edited to match the docstrings rather than regenerated. ## Merge Risk **Door:** two-way; prose and message text only. **Blast Radius:** `docs`, `error-messages`: callers matching on old error strings would break. Companion PRs: lancedb/docs (assembler labels, STYLE.md) and sophon (Enterprise pages). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Wyatt Alt
|
b4cbaf2f67 |
docs: show stateful class Functions
The Functions page only showed a plain function, so it never said where a class's `__init__` runs. Users loading a large model need to know it is not the client: registration reads the signature and ships the source. This adds a section on class Functions: `__init__` runs once per instance on the worker, then `__call__` per row or batch and `close()` on retirement, state is per instance, and `__init__` parameters are initialization fields set per binding. The example's test asserts that registering the class does not construct it. |
||
 JiMyung Lee
|
057a39a1c7 |
docs: row counts in list_indices can be set for remote tables
Remote list_indices fills num_indexed_rows / num_unindexed_rows from the server's enriched response and only leaves them unset for servers that do not report them. The Node and Rust documentation previously implied those fields were always unavailable for remote tables. Synchronize the generated TypeScript API reference with the Node source comments. Closes #4301 Co-authored-by: desty <lee.ji.myung@gmail.com> |
||
 Joshua Snider
|
edb79d38e3 |
docs: correct the documented default retry statuses (#3964)
#2349 changed the default retryable status codes to `[409, 429, 500, 502, 503, 504]`, but only the Rust docstring was updated. The Python and TypeScript bindings — and the generated TypeScript API reference — still document the old `[429, 500, 502, 503]`. The actual default is set in `ResolvedRetryConfig::try_from` in `rust/lancedb/src/remote/retry.rs`, and `rust/lancedb/src/remote/client.rs` already documents it correctly. This brings the other three in line. Docs only — no behavior change. Split out from #3963 per the "one problem at a time" guidance in `CONTRIBUTING.md`. |
||
 J Ganesh Kumar Reddy
|
12d86ea251 |
feat(nodejs): add col() expression builder for camelCase column names
## Summary Fixes #3211 by exposing the shared Rust/DataFusion expression builder through Node bindings. Case-sensitive references such as `col("firstName")` can be composed and passed to `where()`, while SQL strings remain supported. The expression-object representation follows [the design requested on #3290](https://github.com/lancedb/lancedb/pull/3290#discussion_r3632371998). That earlier PR remains open; no claim is made that the issue was unclaimed or that maintainers have selected this implementation. ## Review corrections - Construct the named native struct correctly and parse casts through `arrow_schema::DataType`. - Preserve numeric values: use Int64 only for JavaScript safe integers; retain Float64 otherwise. Explicit `litInt` rejects unsupported numbers with a documented `RangeError`. - Correct test syntax/API calls and align SQL assertions with the shared dialect, including reserved and case-sensitive names. - Use typed, isolated temporary test tables with cleanup, and cover an actual large-number filter threshold. - Document the public methods and commit generated API documentation. Synced with current main `0bceaaf3` through a normal merge. The diff is limited to seven Node source/test/config files and eleven generated documentation files; no Rust core change. ## Validation [Linux validation run](https://github.com/Ganesh1729-ui/lancedb/actions/runs/38244794288) on Node 24 / Rust 1.97.0 with the repository's `ci` profile and `remote,lancedb/metrics-otel` features: - Rebuilt native addon, TypeScript build, Biome, Rust formatting and native Clippy pass. - Expression/query/Arrow/table suites: **1,129 passed**; expression suite repeated: **42 passed**. - Using the original JavaScript numeric dispatch with the rebuilt addon: **8 regressions fail, 2 controls pass**. - Strict TypeDoc generation passes. Final local typing/docs/lint checks pass; the PR source equals the validated source except for the separate validation workflow, which is excluded from this PR. Full provider/model/S3, other feature combinations and OS/Node matrices were not run in this follow-up. Upstream checks and maintainer review remain separate. ## AI assistance AI was used for implementation, review and testing. No prior human review is claimed. |
||
 Lance Release
|
ece54d8917 | Bump version: 0.41.0-beta.16 → 0.41.0-beta.17 | ||
 Brendan Clement
|
882be89778 |
docs: clarify how to contribute website documentation
The contributor guide mixes up shared documentation, Enterprise placeholders and the SDK reference. It also describes section merging that the assembler doesn't do. This replaces the overlapping instructions with guidance on where to write, how whole-page replacement works, how to preview changes and how to keep unfinished docs hidden. It explains that source changes enter a saved release edition before they appear on staging, and points SDK contributors to the existing reference guide. |
||
|
659ea11ac4 |
fix(embeddings): default Cohere to embed-multilingual-v3.0, the v2.0 models are retired (#4698)
Cohere retired `embed-english-v2.0`, `embed-english-light-v2.0` and `embed-multilingual-v2.0` on April 4, 2026 ([deprecations](https://docs.cohere.com/docs/deprecations), which lists `embed-english-v3.0`, `embed-multilingual-v3.0` and `embed-v4.0` as the embedding alternatives). `CohereEmbeddingFunction` still defaulted to `embed-multilingual-v2.0`, so every call without an explicit `name` fails. The default is now `embed-multilingual-v3.0`, the multilingual v3 model, and the retired models are removed from the docstring and the Cohere integration page (the page also listed `embed-english-v2.0` as the default, which did not match the code). The docs example, its generated snippet, the docs test and the slow Cohere test use the new name. This changes the default vector size: `ndims()` returns 1024 for `embed-multilingual-v3.0`, where the old default gave 768. Tables created from a schema that uses `cohere.ndims()` get 1024-dimension vectors from now on. A table that already holds 768-dimension vectors from the retired model needs a re-embed with a current model, since the old model can no longer be called. The `ndims()` entries for the v2 names are left as they are. Test: `test_cohere_default_model_is_not_a_retired_v2_model` in `python/python/tests/test_embeddings.py` checks the default name and that `ndims()` is 1024. The repository's pytest session could not be run for this change (its `conftest.py` needs a newer native module than the released wheel); the body of that test was run against the released `lancedb` 0.40.0 with this version of `cohere.py` in place, where it passes, and it returns `embed-multilingual-v2.0` / 768 without the change. ruff check and format are clean on the touched Python files. No Cohere request was made. This change was made with Claude Code. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
Brendan Clement
|
4359688f32 |
ci: let the Vale action install its pinned CLI (#4708)
Let the pinned Vale action install Vale 3.24.0 instead of maintaining a separate download and extraction step |
||
|
Brendan Clement
|
149cf1052a |
fix(ci): bundle Vale rules to avoid private release downloads (#4707)
The prose check fails because it downloads its Vale rules from the now-private docs repository. Keep the five unchanged rules from `vale-lancedb-v0.1.0` in this repository and disable `vale sync` in CI. This also works for contributors without access to the docs repository. The update instructions record the release and archive checksum. |
||
|
Lance Release
|
45f987a29b | Bump version: 0.41.0-beta.15 → 0.41.0-beta.16 | ||
|
Wyatt Alt
|
bb6aa1a147 |
feat: add a cascade option to drop_database (#4697)
The remote catalog could only drop an empty database: it always asked
the server for a restricted drop.
`DropDatabaseRequest::cascade`, `drop_database(..., cascade=True)` in
Python and `dropDatabase(name, { cascade: true })` in TypeScript ask the
server to drop everything in the database first. The name stays taken
until the server finishes cleaning up the dropped objects' storage. A
restricted drop remains the default.
|
||
|
Jack YeandXuanwo
|
45dbf5eff5 |
fix: use SQL for all remote materialized view APIs (#4673)
Remote materialized-view APIs now use SQL for the full lifecycle. Create and refresh already submit SQL; this change moves describe, paginated listing, and drop onto `DESCRIBE MATERIALIZED VIEW`, `SHOW MATERIALIZED VIEWS`, and `DROP MATERIALIZED VIEW`. Async create, refresh, and drop return the existing Job handle with `job_type = "query"`. The handle has one `id`: the connection-scoped SQL query ID, also accepted by `describe_query`. Submission starts the query, the job drains its result once, and repeated waits reuse the completed result. Synchronous drop waits for catalog removal; physical cleanup proceeds in the background. Listing uses namespace-scoped keyset pagination with opaque resume tokens. Streamed and deferred SQL responses share the same path. Adds transport coverage and updates Python/TypeScript documentation for query-backed drop jobs. --------- Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
Jay PokaleandXuanwo
|
f6a348ac5b |
fix(python): keep FTS scores in RRF and MRR rerankers with return_score="all" (#4677)
`RRFReranker.rerank_hybrid` and `MRRReranker.rerank_hybrid` merge the
vector and FTS results with the base `merge_results`, which keeps only
the first copy of a duplicated row. With `return_score="all"`, a row
that both searches found therefore comes back with `_score = None`, even
though the FTS search scored it. `LinearCombinationReranker` already
copies the FTS score onto such rows.
Reproduction against `lancedb` 0.40.0:
```python
import pyarrow as pa
from lancedb.rerankers import RRFReranker
vector = pa.table({"_rowid": pa.array([1, 2], pa.uint64()), "text": ["a", "b"],
"_distance": pa.array([0.1, 0.2], pa.float32())})
fts = pa.table({"_rowid": pa.array([2, 3], pa.uint64()), "text": ["b", "c"],
"_score": pa.array([5.0, 4.0], pa.float32())})
RRFReranker(return_score="all").rerank_hybrid("q", vector, fts).to_pylist()
# row 2: _distance=0.2, _score=None (expected _score=5.0)
```
`table.search(query_type="hybrid")` is not affected, because
`_combine_hybrid_results` restores the original `_distance` and `_score`
by `_rowid` after reranking. The wrong value shows up when
`rerank_hybrid` is called directly, for example from a custom reranker
that wraps RRF or MRR.
The fix uses `_merge_and_keep_scores` when `return_score="all"`, the
same pattern the model-based rerankers (`CohereReranker`,
`CrossEncoderReranker`, `JinaReranker`, `OpenaiReranker`,
`VoyageAIReranker`, ...) already use. The relevance-only path is
unchanged.
Tests: added `test_rank_fusion_return_score_all_keeps_both_scores` for
both rerankers. It fails without the fix and passes with it. I ran it
against the released 0.40.0 package with this branch's
`rerankers/{base,rrf,mrr}.py` dropped in; `rrf.py` and `mrr.py` on
`main` are identical to 0.40.0. The other reranker tests give the same
results with and without the change. `ruff format` and `ruff check` are
clean.
I used an AI assistant (Claude) while working on this; I reviewed and
tested the change.
Co-authored-by: Xuanwo <github@xuanwo.io>
|
||
|
lancedb-gatefixer[bot]andXuanwo
|
ac99ab173b |
fix(python): support native blob conversion in pandas and batches (#4647)
Plain local blob v2 scans tried to open a pylance dataset before reaching LanceDB's byte-fetch path, so `to_pandas(blob_mode="bytes")` failed when the optional dependency was absent. Batch readers also had no `blob_mode` option. Route supported blob v2 pandas conversions through the native Arrow path. Add `blob_mode` to sync and async batch readers, convert payloads during iteration, and expose the converted schema without fetching the first batch. Apply conversion after reranking and retain the default descriptor behavior. Clean up scanner error punctuation and remove the misleading plain-scan suggestion. Bytes mode retains the existing `to_arrow` restrictions for legacy/list blobs, remote tables, and namespace query pushdown. Lazy pandas handles and legacy pandas scans still require pylance; sync FTS/hybrid batching remains unsupported. Validation: - UV-managed editable Python development build completed successfully. - The absent-pylance regression tests fail against the original query implementation and pass with this change. - Blob and query suites: 403 passed, 1 skipped, 1 deselected. - Remote query suite: 190 passed. - Python-tree Ruff lint/format checks and the async reader Pyright check pass. Repository-wide Ruff reports existing errors outside the Python tree. Fixes #4645 <!-- lance-gatekeeper-fix:v1 agent=fd85998f14bb0b3c92b2d050df93a615 generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
 Dan Tasse
|
6e4f523e14 |
docs: remove logical-column from docs (#4693)
We've removed this feature from LanceDB Enterprise and no longer privilege the "lancedb:logical-column" key. So, removing that from these docs. |
||
|
Xuanwo
|
b44a0d2362 |
chore: update Lance to v15.0.0-beta.3 (#4692)
Upgrade Lance to 15.0.0-beta.3 so `Dataset::concat_data_file_parts` returns a `Send` future and downstream workers can run it on Tokio. The upstream fix is https://github.com/lance-format/lance/pull/10370. Keep the vendored Arrow validity fix; upstream Arrow sources are unchanged between beta.2 and beta.3. |
||
 dependabot[bot]
|
d44609b6f2 |
chore(deps): bump the nodejs-deps group across 1 directory with 3 updates (#4661)
Bumps the nodejs-deps group with 3 updates in the /nodejs directory: [@opentelemetry/api](https://github.com/open-telemetry/opentelemetry-js), [@opentelemetry/sdk-metrics](https://github.com/open-telemetry/opentelemetry-js) and [ts-jest](https://github.com/kulshekhar/ts-jest). Updates `@opentelemetry/api` from 1.9.1 to 1.10.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/open-telemetry/opentelemetry-js/releases">@opentelemetry/api's releases</a>.</em></p> <blockquote> <h2>api/v1.10.0</h2> <h2>1.10.0</h2> <h3>🚀 Features</h3> <ul> <li>feat(api): integrate the Logs API into <code>@opentelemetry/api</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/4862">#4862</a> <a href="https://github.com/hectorhdzg"><code>@hectorhdzg</code></a></li> <li>feat(api): add imperative <code>context.attach()</code> for setting context across callback boundaries that <code>with()</code> cannot wrap. <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6845">#6845</a> <a href="https://github.com/pichlermarc"><code>@pichlermarc</code></a> <ul> <li><code>attach()</code> returns a <code>Token</code> whose <code>dispose()</code> method restores the previous context. Use <code>token.dispose()</code> to detach.</li> <li><code>attach</code> is an optional method on the <code>ContextManager</code> interface; when the active context manager does not implement it, <code>context.attach()</code> logs a warning and returns a no-op token.</li> </ul> </li> <li>feat(api): add <code>AnyValue</code> type (of type <code>unknown</code>) and extend <code>Attributes</code> type values to use <code>AnyValue</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6780">#6780</a> <a href="https://github.com/trentm"><code>@trentm</code></a> <ul> <li>This is part of supporting <a href="https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/4485-extending-attributes-to-support-complex-values.md#how">OTEP 4485</a>.</li> <li>This is a <strong>breaking change for SDK implementations</strong>, in that they may need to support more attribute types for any APIs that accept <code>Attributes</code> or <code>AnyValue</code>. Implementors should refer to the OTEP 4485 link above for "MUST / SHOULD" requirements for complex attributes handling. See also the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/packages/opentelemetry-core/src/common/attributes.ts">attributes-handling utilities provided by <code>@opentelemetry/core</code> to help</a>. For example, <code>cleanSimpleAttributes()</code> can be used to limit user-provided attributes to the set of value types supported in earlier <code>@opentelemetry/api</code> versions.</li> <li>This change is <strong>typically not a breaking change for users</strong> of <code>@opentelemetry/api</code>. There is one possible breaking case: If a TypeScript user was using the <code>Attributes</code> type in their own code and using <a href="https://www.typescriptlang.org/docs/handbook/2/narrowing.html">TypeScript Narrowing</a> to reduce an attribute value to an expected type (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6579#issuecomment-4345316200">example here</a>). It is expected that this usage should be rare and, likely, limited to SDK implementations, if at all. (If this impacted your code, <a href="https://github.com/open-telemetry/opentelemetry-js/issues/new">please let us know</a>.)</li> <li><strong>Note for users of attributes:</strong> Simple attributes SHOULD be used whenever possible. Instrumentations SHOULD assume that backends do not index individual properties of complex attributes, that querying or aggregating on such properties is inefficient and complicated, and that reporting complex attributes carries higher performance overhead.</li> </ul> </li> </ul> <h3>🐛 Bug Fixes</h3> <ul> <li>fix(api): cap tracestate list-members and total length in the deprecated <code>TraceState.set()</code> so it stays in step with <code>TraceState</code> from <code>@opentelemetry/core</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6964">#6964</a> <a href="https://github.com/Zuhef"><code>@Zuhef</code></a></li> <li>fix: resolve the ESM build through the <code>default</code> export condition instead of <code>import</code>, so a resolver that activates neither <code>import</code> nor <code>require</code> still finds an entry point <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7132">#7132</a> <a href="https://github.com/overbalance"><code>@overbalance</code></a></li> </ul> <h3>📚 Documentation</h3> <ul> <li>docs(api): complete <code>@since</code> annotations for the public API <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7023">#7023</a> <a href="https://github.com/nabeelamjadsheikh"><code>@nabeelamjadsheikh</code></a> <ul> <li>Adds the three missing exported-type annotations and the member-level annotations for methods and properties introduced after their enclosing type.</li> <li>Corrects <code>Attributes</code>, <code>AttributeValue</code> and <code>TracerOptions</code> from <code>1.3.0</code> to <code>1.1.0</code>; all three were already importable in 1.1.0 via wildcard re-export.</li> </ul> </li> </ul> <h3>🏠 Internal</h3> <ul> <li>build: migrate the package build from <code>tsc</code> to <code>tsdown</code>, emitting dual CJS/ESM output from a single <code>dist/</code> directory <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6293">#6293</a> <a href="https://github.com/overbalance"><code>@overbalance</code></a> <ul> <li>The public entry points are unchanged: <code>@opentelemetry/api</code> and <code>@opentelemetry/api/experimental</code>. This package already declared an <code>exports</code> map, so deep imports such as <code>@opentelemetry/api/build/src/...</code> were rejected before this change and continue to be.</li> <li>The emitted files moved out of <code>build/src</code> (CJS), <code>build/esm</code> and <code>build/esnext</code> (ESM) into <code>dist/</code>, using <code>.cjs</code>/<code>.mjs</code> extensions with matching <code>.d.cts</code>/<code>.d.mts</code> declarations. The non-standard <code>esnext</code> condition has been dropped in favour of the standard <code>import</code>/<code>require</code> conditions.</li> </ul> </li> <li>perf(api): add getGlobal fast-path <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6956">#6956</a> <a href="https://github.com/legendecas"><code>@legendecas</code></a></li> <li>chore: remove the redundant <code>module</code> field from <code>package.json</code>; bundlers resolve the ESM build through the <code>exports</code> map <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7130">#7130</a> <a href="https://github.com/overbalance"><code>@overbalance</code></a></li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md">@opentelemetry/api's changelog</a>.</em></p> <blockquote> <h2>1.10.0</h2> <h3>🚀 (Enhancement)</h3> <ul> <li>feat(resource): create sync resource with some attributes that resolve asynchronously <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3460">#3460</a> <a href="https://github.com/samimusallam"><code>@samimusallam</code></a></li> <li>feat: collect host id for non-cloud environments <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3575">#3575</a> <a href="https://github.com/mwear"><code>@mwear</code></a></li> <li>feat (api-logs): separate Events API into its own package <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3550">3550</a> <a href="https://github.com/martinkuba"><code>@martinkuba</code></a></li> <li>feat(sdk-metrics): apply binary search in histogram recording <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3539">#3539</a> <a href="https://github.com/legendecas"><code>@legendecas</code></a></li> <li>perf(propagator-jaeger): improve deserializeSpanContext performance <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3541">#3541</a> <a href="https://github.com/doochik"><code>@doochik</code></a></li> <li>feat: support TraceState in SamplingResult <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3530">#3530</a> <a href="https://github.com/raphael-theriault-swi"><code>@raphael-theriault-swi</code></a></li> <li>feat(sdk-trace-base): add diagnostic logging when spans are dropped <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3610">#3610</a> <a href="https://github.com/neoeinstein"><code>@neoeinstein</code></a></li> <li>feat: add unit to view instrument selection criteria <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3647">#3647</a> <a href="https://github.com/jlabatut"><code>@jlabatut</code></a></li> <li>feat(tracing): expose dropped counts for attributes, events and links on span <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3576">#3576</a> <a href="https://github.com/mohitk05"><code>@mohitk05</code></a></li> </ul> <h3>🐛 (Bug Fix)</h3> <ul> <li>fix(core): added falsy check to make otel core work with browser where webpack config had process as false or null <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3613">#3613</a> <a href="https://github.com/ravindra-dyte"><code>@ravindra-dyte</code></a></li> <li>fix(instrumentation-http): include query params in http.target <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3646">#3646</a> <a href="https://github.com/kobi-co"><code>@kobi-co</code></a></li> </ul> <h3>📚 (Refine Doc)</h3> <ul> <li>chore: update http example <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3651">#3651</a> <a href="https://github.com/JamieDanielson"><code>@JamieDanielson</code></a></li> </ul> <h3>🏠 (Internal)</h3> <ul> <li>chore(exporter-jaeger): deprecate jaeger exporter <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3585">#3585</a> <a href="https://github.com/pichlermarc"><code>@pichlermarc</code></a></li> <li>fix(sdk-metrics): fix flaky LastValueAggregator test by using fake timer <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3587">#3587</a> <a href="https://github.com/pichlermarc"><code>@pichlermarc</code></a></li> <li>fix(test): fix failing tests by preventing source-map generation <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3642">#3642</a> <a href="https://github.com/pichlermarc"><code>@pichlermarc</code></a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/56e6b1bb890f844b8963a146780d0b9cfa8abd0d"><code>56e6b1b</code></a> chore: Release API 1.4.1, SDK 1.10.0, Experimental 0.36.0 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3654">#3654</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/ecb5ebe86eebc5a1880041b2523adf5f6d022282"><code>ecb5ebe</code></a> fix: Minor typing issues for web sandbox (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3660">#3660</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/48ecf229eb0b468cf433a152daf414313b381edc"><code>48ecf22</code></a> feat(sdk-trace-base): add dropped attributes and events count on span (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3576">#3576</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/95bf6fc7cc105469def055f1f93446c9cd09882d"><code>95bf6fc</code></a> fix(build): add <code>example/http</code> to ignore list in <code>update-ts-configs</code> (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3657">#3657</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/3a2ca58075cc7e0eea8233c51735db5d583c11d0"><code>3a2ca58</code></a> feat: add unit to view instrument selection criteria (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3647">#3647</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/ebc8575a0c215a7b7c84866116a261ffd8aff015"><code>ebc8575</code></a> chore(jeager-exporter): deprecate jaeger exporter (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3585">#3585</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/2f715bd78dd7fb66e2aa2b76fb6e7c623deddd60"><code>2f715bd</code></a> feat: allow to add attributes from grpc metadata in the patched server (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3589">#3589</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/cbbdfd379854fcb41bd70841ae4890200a2fda09"><code>cbbdfd3</code></a> fix(sdk-node): update instrumentations once MeterProvider is initialized (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3624">#3624</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/4a4484a910201466121ed45fa12566737f6ac19d"><code>4a4484a</code></a> chore: update http example (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3651">#3651</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/494efffcf4648e3b1207c3c925c1a52bcc7f7350"><code>494efff</code></a> fix(otlp-exporter-base): fix changelog for merged retry pr (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3650">#3650</a>)</li> <li>Additional commits viewable in <a href="https://github.com/open-telemetry/opentelemetry-js/compare/v1.9.1...v1.10.0">compare view</a></li> </ul> </details> <br /> Updates `@opentelemetry/sdk-metrics` from 2.11.0 to 2.12.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/open-telemetry/opentelemetry-js/releases">@opentelemetry/sdk-metrics's releases</a>.</em></p> <blockquote> <h2>v2.12.0</h2> <h2>2.12.0</h2> <h3>🚀 Features</h3> <ul> <li>chore(sdk-trace): add <code>StackContextManager</code> in <code>@opentelemetry/sdk-trace</code> package <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7086">#7086</a></li> </ul> <h3>🏠 Internal</h3> <ul> <li>feat(ci): support releasing from maintenance branches <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6767">#6767</a> <a href="https://github.com/pichlermarc"><code>@pichlermarc</code></a> <ul> <li>The API documentation site is only redeployed for releases whose commit is reachable from <code>main</code>, so a maintenance release no longer overwrites it.</li> </ul> </li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md">@opentelemetry/sdk-metrics's changelog</a>.</em></p> <blockquote> <!-- raw HTML omitted --> <h1>CHANGELOG</h1> <p>All notable changes to this project will be documented in this file.</p> <p>For API changes, see the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/api/CHANGELOG.md">API CHANGELOG</a>. For experimental package changes, see the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/experimental/CHANGELOG.md">experimental CHANGELOG</a>. For semantic convention package changes, see the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/semantic-conventions/CHANGELOG.md">semconv CHANGELOG</a>. For notes on migrating to 2.x / 0.200.x see <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/upgrade-to-2.x.md">the upgrade guide</a>. For notes on migrating to 3.x see <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/3.x/migration-guide.md">the 3.x migration guide</a>.</p> <h2>Unreleased</h2> <h3>💥 Breaking Changes</h3> <h3>🚀 Features</h3> <h3>🐛 Bug Fixes</h3> <h3>📚 Documentation</h3> <h3>🏠 Internal</h3> <h2>3.0.0</h2> <h3>💥 Breaking Changes</h3> <ul> <li>chore(sdk-trace-node)!: remove the <code>@opentelemetry/sdk-trace-node</code> package <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7054">#7054</a> <ul> <li>The sdk-trace-node package has been replaced by the <code>@opentelemetry/sdk-trace</code> package. See the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/3.x/migration-guide.md">3.x migration guide</a> for full migration instructions.</li> </ul> </li> <li>chore(shim-opentracing)!: remove the <code>@opentelemetry/shim-opentracing</code> package <ul> <li>In the <a href="https://github.com/open-telemetry/opentelemetry-specification/releases/tag/v1.56.0">OpenTelemetry Specification v1.56.0</a> the <a href="https://redirect.github.com/open-telemetry/opentelemetry-specification/pull/4938">OpenTracing compatibility requirements were deprecated</a>. The JavaScript OpenTracing shim package will not receive any more releases after the current <a href="https://www.npmjs.com/package/@opentelemetry/shim-opentracing/v/2.12.0">2.12.0 release</a> (<a href="https://github.com/open-telemetry/opentelemetry-js/tree/v2.12.0/packages/opentelemetry-shim-opentracing/">source code for last release</a>).</li> </ul> </li> <li>chore(context-async-hooks)!: remove the unused class <code>AsyncHooksContextManager</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7078">#7078</a></li> <li>feat!: migrate package builds from <code>tsc</code> to <code>tsdown</code>, emitting dual CJS/ESM output from a single <code>dist/</code> directory and declaring an <code>exports</code> map on every package <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6293">#6293</a> <a href="https://github.com/overbalance"><code>@overbalance</code></a> <ul> <li>Importing a package by its name is unaffected in both CommonJS and ESM, as is every subpath listed in its <code>exports</code> map.</li> <li><strong>Deep imports into the build output no longer resolve.</strong> An <code>exports</code> map is an allowlist that Node.js and bundlers enforce, so specifiers such as <code>@opentelemetry/core/build/src/...</code> or <code>@opentelemetry/core/build/esm/...</code> now fail with <code>ERR_PACKAGE_PATH_NOT_EXPORTED</code>. Rewriting them to the new file layout does not help — unlisted subpaths are rejected whether or not the file exists.</li> <li>The emitted files moved out of <code>build/src</code> (CJS), <code>build/esm</code> and <code>build/esnext</code> (ESM) into <code>dist/</code>, using <code>.cjs</code>/<code>.mjs</code> extensions with matching <code>.d.cts</code>/<code>.d.mts</code> declarations.</li> <li><code><package>/package.json</code> is no longer importable for the same reason. Read a package's version from your own dependency metadata, or use <code>SDK_INFO</code> from <code>@opentelemetry/core</code> for the SDK version.</li> <li>The non-standard <code>esnext</code> entry point has been removed; tools that preferred it fall back to <code>module</code> (ESM) or <code>main</code> (CJS).</li> <li>If you depend on something that is only reachable through a deep import, please open an issue so it can be considered for the public API.</li> </ul> </li> <li>feat(core)!: remove deprecated <code>getTimeOrigin</code>, <code>otperformance</code>, <code>_globalThis</code>, and <code>unrefTimer</code> from <code>@opentelemetry/core</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7053">#7053</a> <ul> <li><code>getTimeOrigin()</code> — use <code>performance.timeOrigin</code> directly.</li> <li><code>otperformance</code> — use the global <code>performance</code> object directly.</li> <li><code>_globalThis</code> — use <code>globalThis</code> directly.</li> <li><code>unrefTimer(timer)</code> — call <code>timer.unref()</code> directly in your own code.</li> </ul> </li> <li>feat(sdk-trace)!: remove deprecated <code>TracerProviderOptions.forceFlushTimeoutMillis</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7057">#7057</a> <ul> <li>Pass <code>timeoutMillis</code> to <code>provider.forceFlush({ timeoutMillis })</code> instead. The default timeout is 30000ms.</li> </ul> </li> <li>feat(propagator-jaeger)!: remove <code>@opentelemetry/propagator-jaeger</code> package <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7077">#7077</a> <ul> <li>The Jaeger propagator is deprecated by the OpenTelemetry specification in favour of <code>W3CTraceContextPropagator</code>. Use <code>W3CTraceContextPropagator</code> from <code>@opentelemetry/core</code> instead.</li> <li>See the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/3.x/migration-guide.md">3.x migration guide</a> for full instructions.</li> </ul> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/92b844e8aa7d9655e4b3560154e912c4c1ced64d"><code>92b844e</code></a> chore: prepare next v2.x release (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7151">#7151</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/a859aff9825ec8138cf287d08ac86401f00aad7c"><code>a859aff</code></a> fix(web-common): export getElementXPath utility (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7124">#7124</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/2bd5a73e7af7ab363f3b239f8cb1f5f56eaa3ad5"><code>2bd5a73</code></a> chore(web-common)!: deprecate <code>addSpanNetworkEvents()</code> et al instrumentation-...</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/d60f05edeac6aec8e64736c10672077fd4a766b5"><code>d60f05e</code></a> doc(sdk-trace): add description for StackContextManager (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7089">#7089</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/5bd476e6f7a22e6cf2e94a645833247284aa295d"><code>5bd476e</code></a> chore: move context manager to sdk-trace (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7086">#7086</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/554c0c6467c77e9ef1f2676f2bd20b505e228417"><code>554c0c6</code></a> chore: move sdk-trace-web utils to web-common (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7058">#7058</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/788922c5780a17d5aaace76a0da458858b2ad88a"><code>788922c</code></a> ci: support maintenance branch releases (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7036">#7036</a>)</li> <li>See full diff in <a href="https://github.com/open-telemetry/opentelemetry-js/compare/v2.11.0...v2.12.0">compare view</a></li> </ul> </details> <br /> Updates `ts-jest` from 29.4.12 to 29.4.15 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/kulshekhar/ts-jest/releases">ts-jest's releases</a>.</em></p> <blockquote> <h2>v29.4.15</h2> <p>Please refer to <a href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a> for details.</p> <h2>v29.4.14</h2> <p>Please refer to <a href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a> for details.</p> <h2>v29.4.13</h2> <p>Please refer to <a href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a> for details.</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">ts-jest's changelog</a>.</em></p> <blockquote> <h2><a href="https://github.com/kulshekhar/ts-jest/compare/v29.4.14...v29.4.15">29.4.15</a> (2026-10-15)</h2> <h3>Bug Fixes</h3> <ul> <li><strong>compiler:</strong> resolve modules with tsconfig options regardless of cache state (<a href="https://github.com/kulshekhar/ts-jest/commit/268746c0fcaf72476cebb348c5543e5c2aac3622">268746c</a>), closes <a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5216">#5216</a></li> <li>support <code>.cjs</code> config files in <code>config:migrate</code> (<a href="https://github.com/kulshekhar/ts-jest/commit/4001b4e6d28aa461e7468f09157413e113f0bd92">4001b4e</a>), closes <a href="https://redirect.github.com/kulshekhar/ts-jest/issues/4485">#4485</a></li> </ul> <h2><a href="https://github.com/kulshekhar/ts-jest/compare/v29.4.13...v29.4.14">29.4.14</a> (2026-09-25)</h2> <h3>Reverts</h3> <ul> <li>bring back prior changes of compiler util (<a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5471">#5471</a>) (<a href="https://github.com/kulshekhar/ts-jest/commit/01e5a2e17ac9c940880865d6a81751832dcc4184">01e5a2e</a>), closes <a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5469">#5469</a></li> </ul> <h2><a href="https://github.com/kulshekhar/ts-jest/compare/v29.4.12...v29.4.13">29.4.13</a> (2026-09-22)</h2> <h3>Features</h3> <ul> <li>support Babel 8 (<a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5466">#5466</a>) (<a href="https://github.com/kulshekhar/ts-jest/commit/c3a0b57c00d8b6acfb72b020e2411752d39073e2">c3a0b57</a>), closes <a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5412">#5412</a></li> </ul> <h3>Performance Improvements</h3> <ul> <li><strong>compiler:</strong> cache module resolution modes (<a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5419">#5419</a>) (<a href="https://github.com/kulshekhar/ts-jest/commit/697d337c1b79a0804972a3d30c0875b02e73bf62">697d337</a>), closes <a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5402">#5402</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/kulshekhar/ts-jest/commit/197d4de96002589658d8a6c2d3b9b9554d17af6b"><code>197d4de</code></a> chore(release): 29.4.15</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/ebad800d2c080cb72d7f0ea3225221b0cee066da"><code>ebad800</code></a> build(deps): update dependency node to v24.22.0</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/e5f6e986fe7c9e8115a7afe49ec9d5e8a576e93b"><code>e5f6e98</code></a> test(compiler): clean up temp dirs from module resolution tests</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/77dbb184d5895f8e315d574971fc83ab6a8a43b2"><code>77dbb18</code></a> test(compiler): make module resolution tests path-independent on Windows</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/268746c0fcaf72476cebb348c5543e5c2aac3622"><code>268746c</code></a> fix(compiler): resolve modules with tsconfig options regardless of cache state</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/15362016c490ae230c3d69f7a9399513d3600ed7"><code>1536201</code></a> build(deps): update dependency vite to ^8.3.4</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/141e39c092759ca6173095fe2dd4ebcdd3117e62"><code>141e39c</code></a> build(deps): update dependency <code>@babel/core</code> to ^8.0.7</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/6a6686e5374525db09ac67f37cf0f2d53c372f28"><code>6a6686e</code></a> build(deps-dev): bump shell-quote from 1.10.0 to 1.12.0 in /examples</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/98e6e50fe934e8e86c903719048891a3a071a2b7"><code>98e6e50</code></a> build(deps): bump proxy-addr from 2.0.7 to 2.0.8 in /website</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/d2439ff5ed9f6425bed60d0202f37771022f0526"><code>d2439ff</code></a> build(deps): bump compression from 1.8.1 to 1.8.2 in /website</li> <li>Additional commits viewable in <a href="https://github.com/kulshekhar/ts-jest/compare/v29.4.12...v29.4.15">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
|
0cccc9c991 |
chore(deps): bump rust-minor-patch group and napi 3.15 (#4669)
Bumps the rust-minor-patch group: serde_json 1.0.154, tokio 1.54.1, uuid 1.28.0, reqwest 0.13.5, webbrowser 1.2.5, napi-derive 3.6.13, napi-build 2.6.0, arc-swap 1.9.3, libc 0.2.191. On top of the Dependabot update: * napi is bumped to 3.15.0 instead of 3.13.0. napi-derive 3.6.13 generates code that references `bindgen_prelude::NativeBorrowScopeRelease`, which napi 3.13.0 doesn't have, so the nodejs crate failed to compile. The napi and napi-derive floors in `nodejs/Cargo.toml` are raised so they can't drift apart again. * The RUSTSEC-2026-0410 ignore from #4683 is removed from `deny.toml`. #4690 already moved the lockfile to zstd-sys 2.1.2, which fixes it. The other three zstd ignores stay until we can move to Arrow 59+. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Will Jones <willjones127@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
lancedb-gatefixer[bot]
|
fe079eb421 | fix!: restore blob SQL inserts with explicit adapter schemas (#4687) | ||
 Ozawa SongandXuanwo
|
4b373b2070 |
fix(python)!: support compat-only imports (#3973)
Read `__version__` from the native `_lancedb` module so `import lancedb` works when only `lancedb-compat` is installed. Add the type declaration and a subprocess regression test. Ruff and Linux-targeted Pyright pass. Import tests: 1 passed and 1 Linux-only test skipped on Windows. Runtime tests used the official same-version Windows native module; a fresh source build still needs CI. BREAKING CHANGE: `lancedb.__version__` now uses the native version string. Prereleases may report `0.41.0-beta.13` instead of `0.41.0b13`. Fixes #3950 --------- Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
Jay PokaleandXuanwo
|
dcac834233 |
fix(python)!: reject tuple fields with mixed element types in pydantic models (#4679)
`pydantic_to_schema` maps a `Tuple[...]` field to an Arrow list of its
**first** element type. A tuple with different element types gets the
wrong schema, and the error only surfaces when a row is written:
```python
class Pair(LanceModel):
pair: Tuple[int, str]
pydantic_to_schema(Pair) # pair: list<item: int64>
db.create_table("t", schema=Pair).add([Pair(pair=(1, "a"))])
# pyarrow.lib.ArrowInvalid: Could not convert 'a' with type str: tried to convert to int64
```
An Arrow list has a single element type, so such a tuple cannot be
stored as one. This raises a `TypeError` when the schema is built,
saying the tuple elements must share one type and pointing to a nested
`LanceModel` for a fixed set of differently typed fields. Homogeneous
tuples are unchanged: `Tuple[int, int]` and `Tuple[float, ...]` still
map to lists.
**Breaking change:** a model with a mixed-type tuple field, which used
to get a (wrong) `list<...>` schema from `pydantic_to_schema`, now
raises `TypeError` when the schema is built. Such a schema could not
hold the model's values: writing a row failed with `ArrowInvalid`. The
requirement and the nested-`LanceModel` alternative are documented in
the `pydantic_to_schema` docstring.
Tests (in `test_pydantic.py`): `TypeError` for `Tuple[int, str]` (fails
on `main`), and the list mapping for `Tuple[int, int]` and `Tuple[float,
...]`. The whole file passes; `ruff format` and `ruff check` are clean.
I used an AI assistant (Claude) while working on this; I reviewed and
tested the change.
---------
Co-authored-by: Xuanwo <github@xuanwo.io>
|
||
 陈志谦andXuanwo
|
3575f972ac |
fix(python): accept tuples and pyarrow arrays as hybrid search vectors (#4662)
The synchronous hybrid query builder rejected vector containers that the
rest of the Python API accepts:
```python
table.search(query_type="hybrid").vector((1.0, 2.0)).text("hello").to_pandas() # ValueError
table.search(query_type="hybrid").vector(pa.array([1.0, 2.0], pa.float32()))...
# ValueError: Vector query must be either a string or a vector
```
`lancedb.common.VEC` documents `list[float] | tuple[float, ...] |
np.ndarray | pa.Array`, and the same two containers work for
`table.search(vector)` and for the async hybrid API
(`AsyncQuery.nearest_to(...).nearest_to_text(...)`).
`LanceHybridQueryBuilder._validate_query` only listed `str | list |
np.ndarray`, so a tuple or a pyarrow array was refused before reaching
the query planner.
This normalises the vector exactly the way `LanceQueryBuilder.create`
already does — list/tuple to a `np.float32` array, `np.ndarray` through
`astype(float32)`, `pa.Array`/`pa.ChunkedArray` through `np.array(...,
dtype=float32)`, strings passed through — and keeps the existing
`ValueError` for anything else.
`python/python/tests/test_hybrid_query.py::test_hybrid_query_accepts_every_vector_container`
is parametrised over tuple, pyarrow array and pyarrow chunked array, and
asserts the returned `_rowid`s match the `list` form: 3 failed on `main`
(`ValueError` from `query.py:2452`), 3 passed with the change.
Neighbouring suites (`test_hybrid_query.py`, `test_query.py`,
`test_rerankers.py`) are 220 passed / 9 skipped, and `test_table.py`,
`test_remote_db.py`, `test_fts.py`, `test_index.py`,
`test_permutation.py` are 724 passed / 5 skipped. `ruff` at the version
pinned in the repository's pre-commit config (0.9.9) reports no findings
and `ruff format --check` is clean.
One source file (`python/python/lancedb/query.py`), no behaviour change
for containers that already worked.
Prepared with AI assistance; every line was verified by running the
cases above.
---------
Co-authored-by: Xuanwo <github@xuanwo.io>
|
||
|
lancedb-gatefixer[bot]andXuanwo
|
2cd635d41c |
fix: correct remote materialized view refresh results and options (#4672)
Remote materialized-view refreshes drained the SQL result and then fabricated `mode=rebuild`, `rows_written=<total view rows>`, and `source_version=0`. This made incremental refreshes report incorrect statistics. Make these three result fields optional across Rust, Python, and TypeScript, and leave them unset for remote SQL refreshes. Remove the post-refresh row count request. Local refreshes continue to populate all metrics, and `version` remains the view version observed after completion. Remote per-operation statistics remain unavailable until the SQL job adapter can retrieve them. Expose the forced-rebuild option through the Rust builder, synchronous/asynchronous Python refresh and job methods, and TypeScript refresh. Remote `full=true` sends `WITH (full = true)`; the default lets the server choose its refresh mode. Local source-version pinning and remote job cancellation are preserved. Local refresh regression tests respect the `experimental-functions` build gate, while remote refresh coverage also runs in default builds. Validation: - Rust: 20 materialized-view tests and 21 remote SQL tests passed with `remote,experimental-functions`; `cargo fmt --all` passed. - Python: rebuilt the extension in the uv-managed environment. The default build passed 23 tests with 27 feature-dependent skips. The opt-in build passed 61 tests with 3 expected default-build-only skips, covering the materialized-view suite and 14 async query-serialization cases from current main. Python Ruff lint and formatting checks passed. - TypeScript: default and opt-in native builds and TypeScript compilation passed. The default build passed 5 tests with 10 expected feature-dependent skips; the opt-in build passed all 14 materialized-view tests with the default-build-only test skipped. Lint/format checks and documentation generation passed. - Repository-wide `ruff check .` reports 48 pre-existing violations in unchanged `docs/web-tests` files; `ruff check python` passes. Fixes #4670 <!-- lance-gatekeeper-fix:v1 agent=ece43ec4b88a48d1789f60005fa3ecfc generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
lancedb-gatefixer[bot]andXuanwo
|
c167b99fa0 |
fix: read legacy blob columns through blob APIs (#4649)
Legacy blob v1 columns were rejected by LanceDB's blob validation and omitted from `blob_columns()`, even though the pinned Lance reader already supports them. Accept both blob versions in read validation and discovery so `fetch_blobs`, `fetch_blob_files`, and `fetch_blob_ranges` can read legacy payloads through the existing native reader. Keep list-nested blobs excluded from row-based reads and document how to fetch legacy blobs using explicit row IDs. Regression coverage checks sync and async APIs, independently written Lance datasets, ordering and duplicates across fragments, null and empty blobs, lazy seeking, ranges, and deleted rows. Fixes #4644 Validation: - Both new Python regressions failed on the original implementation with the reported legacy-column rejection and empty discovery result, then passed with the fix. - Python blob suite: 202 passed, 1 skipped. - Remote blob tests: 29 passed. - Focused Rust blob unit tests with `remote`: 13 passed. - Rust blob integration suite with `remote`: 59 passed, including legacy storage 2.0 and 2.1. - Rust compile and Clippy checks with `remote`, tests, and examples passed. - Python package Ruff lint and format checks passed; Rust formatting passed. - Repository-wide `ruff check .` reports 49 existing errors in unchanged `docs/web-tests` examples; the Python package is clean. <!-- lance-gatekeeper-fix:v1 agent=02b7edcbfa8ff0e7fad4615159249b3e generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
lancedb-gatefixer[bot]andXuanwo
|
83b7275137 |
fix(remote): make HTTP timeout errors actionable (#4650)
Remote `create_table` reader uploads can exceed the read deadline while waiting for response headers, but reqwest's main error message only says "error sending request". Make HTTP and retry errors explicitly identify timeouts and name the relevant `client_config.timeout_config` settings and environment variables, preserving typed sources, request metadata, and retry classification. Document that the read timeout covers waiting for headers during upload. This implements the issue's diagnostic option: creation still uses one streaming request, so large or slow readers may require a longer `client_config.timeout_config.read_timeout`. Validation: - 48 focused Rust client, error, and retry tests passed. Regression coverage holds an upload open until its read timeout and also exercises TLS connection timeouts, with and without retries. - 8 Python create-table and HTTP/retry error tests passed in the uv-managed environment after rebuilding the extension, including reader creation with read and overall timeouts. - `cargo check --quiet --features remote --tests --examples` passed. - `cargo clippy --quiet --features remote --tests --examples` passed. - Rust formatting and Python package Ruff lint/format checks passed. - Repository-wide `ruff check .` reports 49 existing findings under `docs/web-tests`; those files are unchanged. Fixes #4646 <!-- lance-gatekeeper-fix:v1 agent=ce1c040fa581f1748166ecb686d9c9c0 generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
Xuanwo
|
ad1212fb32 |
chore: update Lance to v15.0.0-beta.2 (#4690)
**Blocked on the corrected Lance release:** https://github.com/lance-format/lance/pull/10370 restores the `Send` contract required by downstream Function workers. Refresh this upgrade with that release before merging. Upgrade the Rust, Java, and Python Lance dependencies to https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.2. This brings in bounded vector-index consolidation without retraining (https://github.com/lance-format/lance/pull/10335) and deletion filtering for index entries whose fragment coverage was pruned (https://github.com/lance-format/lance/pull/10325). Synchronize the vendored `lance-arrow` with the release and retain its offset-safe validity patch. The existing `nullable_struct_merge` regression still panics with the unpatched release, so removing that patch would regress nullable nested queries. Refresh both lockfiles, including the `zstd-sys` version required by the new Lance encoding dependency. Update the custom memtable bitmap test adapter to train on row addresses, matching the new bitmap contract; remove its obsolete row-ID conversion. Preserve the Blob APIs' existing missing-row error messages when newer Lance rejects a missing fragment earlier in the read path. Retain upstream attribution in vendored sources while satisfying the repository license-header check. Validation of the compatibility fixes: all 58 Rust Blob integration tests and 219 Python Blob tests pass (one Python test skipped). Strict workspace Clippy, formatting, and diff checks pass. |