mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 14:16:54 +00:00
1b40e479d3ea9d056a259166e4ece9f49056b592
753
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Lance Release
|
3d790e751d | Bump version: 0.42.0-beta.4 → 0.42.0-beta.5 | ||
 Yang Cen
|
9ba1920e0e |
fix(node): normalize record batches before IPC serialization (#4871)
Passing a record batch created by an ESM `apache-arrow` import to `RRFReranker.rerankHybrid` throws an IPC footer parse error. The SDK's CommonJS Arrow writer receives data types from another module instance. Normalize single-record-batch input with the existing table sanitizer before writing IPC. Extend the Arrow 15–21 round-trip test with caller-created batches and nulls, and add a native ESM reranker regression covering 64-bit row IDs, scores, and an empty FTS side. Validation: - Reproduced with published `0.42.0-beta.3`; CommonJS and explicitly normalized ESM controls pass. - Same native module and inputs: all 8 targeted cases fail with the parent serializer and pass with this change. - Local native build, TypeScript build, lint, and formatting pass. - Current-head CI passes on Linux Node 22/24/26, macOS, and Arrow 19/20. The macOS run passes 1,643 tests; Linux Node 24 also passes docs and example checks. |
||
 Xuanwo
|
edc49e60e2 |
feat: expose Function failure tracebacks (#4861)
Function refresh failures and skipped-row diagnostics lose the traceback when decoded by the SDK. Expose the server-provided optional `traceback` on job failure details and Function error records in Rust, Python, and Node.js, while keeping existing compact error messages. Responses from older servers remain supported when the field is absent. Tracebacks may be truncated by the server. The Rust `Error::JobFailed` variant now stores `Box<JobFailure>` so the added diagnostics do not enlarge every `Result` error payload. |
||
|
Lance Release
|
17f90f8403 | Bump version: 0.42.0-beta.3 → 0.42.0-beta.4 | ||
|
Lance Release
|
4db3998b65 | Bump version: 0.42.0-beta.2 → 0.42.0-beta.3 | ||
  Bruno RamirezandClaude Sonnet 5
|
a9ad499781 |
feat: support remote table compaction via SQL OPTIMIZE TABLE statement (#4802)
`Table::optimize()` previously returned `NotSupported` for remote tables. This PR wires it to the LanceDB Enterprise server's `OPTIMIZE TABLE ... REWRITE DATA` SQL statement, submitted over Flight SQL. Only `OptimizeAction::Compact` without a custom `remap_options` or non-default `CompactionOptions` is supported remotely. `OptimizeAction::All`, `::Prune`, and `::Index` all return a clear `NotSupported` error instead of silently doing the wrong thing. A branch-scoped table handle submits a `WITH (branch = '<branch>')` clause; the statement doesn't validate the branch exists itself, so an unknown branch surfaces as an async job failure rather than an error from this call. The server's SQL job adapter doesn't expose compaction metrics, so `OptimizeStats::compaction` is `Some(CompactionMetrics::default())`, a placeholder rather than real numbers — it must still be `Some`, since callers (including the Python/Node bindings) unwrap it unconditionally on success. This mirrors how `refresh_materialized_view_async` already handles the same limitation for its own metrics. The Python and Node `optimize()` convenience methods call a new shared `Table::optimize_full`, which runs `Compact` and then, on a remote table, skips `Prune` and `Index` instead of attempting and failing on them — returning zeroed prune stats rather than submitting a compaction job and then raising. Python's `RemoteTable.optimize()` also now forwards `compaction_options` to the Rust layer, so a non-default value is rejected before submission instead of being silently dropped. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Lance Release
|
dcda5224ee | Bump version: 0.42.0-beta.1 → 0.42.0-beta.2 | ||
|
Lance Release
|
33c62e8f3e | Bump version: 0.42.0-beta.0 → 0.42.0-beta.1 | ||
|
Xuanwo
|
f6612a5f2f |
test(nodejs): preserve Function worker observations in job progress (#4819)
Extend the existing job progress regression to cover Function worker counts, GPU capacity and wait reasons, and known or absent warm-reuse evidence through `Job.progress()` and `Job.wait()`. The callback assertions also preserve an unknown warm state instead of turning it into `false`. Validation on EC2: all 56 remote SDK tests, Biome lint/format, TypeScript build, and documentation generation pass. Native code and dependencies match the previously validated build. Refs ENT-3486. |
||
 XuanwoandEC2 Default User
|
8d10f25121 |
feat(nodejs): expose structured job progress (#4815)
TypeScript job handles now expose structured progress through `progress()` and an optional synchronous callback to `wait()`. Snapshots preserve unknown fields and missing counters, and job listings include progress and pause state. Existing `status()` strings and no-argument `wait()` behavior remain compatible. Wait callbacks observe the existing backend poller. Observer exceptions are logged without changing the job outcome. A best-effort ETA is available only from compatible samples with positive progress, and is omitted during initialization, queueing, or pause. Part of ENT-3500. --------- Co-authored-by: EC2 Default User <ec2-user@ip-172-31-12-104.us-east-2.compute.internal> |
||
 Gaétan
|
51bdfd95f5 |
docs: clarify remote index build completion (#4436)
On remote tables, `create_index` returns as soon as the build is scheduled. Until it finishes, a vector index is missing from `list_indices`, a scalar or FTS index shows 0 indexed rows, and queries don't use the index. The docs didn't say any of this, so it was easy to assume the index was ready. This adds that to the `create_index` and `list_indices` docs in Python (`Table` and `AsyncTable`) and TypeScript, and points to `wait_timeout` / `waitTimeoutSeconds`, `wait_for_index`, and `create_index_async` for blocking until the index is ready. It also explains that completion tracking is tied to the submitted build only when the server returns a job ID, while legacy name-based waits can return early when replacing an existing index. Closes #4405 |
||
|
Xuanwo
|
2d70bec53a |
feat: add UTM-tagged docs links to SDK warnings and enterprise-only errors (#4797)
Add a central documentation link registry (`lancedb::docs::DocLink`) in the Rust core that builds `docs.lancedb.com` URLs with UTM parameters (`utm_source=lancedb_sdk`, `utm_medium=warning|error`, `utm_campaign=<id>`, `utm_content=<sdk>-<version>`) placed before any `#anchor` fragment and without user-identifying data. - Initialize `utm_content` per language binding (`python`, `nodejs`, `cli`, defaulting to `rust`) and expose the link builder to Python so `DBConnection` and `Job` stubs reuse the core definitions. - Append published documentation links to Enterprise-only `NotSupported` / `NotImplementedError` errors across database, job, catalog, and table operations. - Emit rate-limited maintenance warnings (once per table per process when `warn` logging is enabled) when opening a native table with at least 100 small fragments (`< 100k` physical rows each) or at least 1,000,000 unindexed rows on an optimizable user index, reading only manifest metadata without opening data or `_indices/` files, and document the thresholds in the compaction and reindexing guides. - Verify published/in-repo `docs/web/` targets and `#anchor` fragments in unit tests and include extracted SDK links in the scheduled `docs-link-check` workflow. Linear: OSS-2772 |
||
  LeoandLeo
|
f2bcb156ad |
feat: expose approx_mode on vector queries
`ApproxMode` landed in the Rust core in #3549, but the Python and Node.js bindings were never plumbed, so `approx_mode` is unreachable from either SDK. Both were built mirroring the existing `ef` plumbing. Co-authored-by: Leo <leo@insight.co> |
||
 lancedb-gatefixer[bot]
|
c18aa7b033 |
fix: reject renaming or dropping embedding columns
Renaming or dropping an embedding source or vector column leaves schema metadata pointing at missing fields, causing later `add()` calls to fail. Reject these operations in local schema evolution before committing any changes. The guard reads Python/Node embedding column metadata and native Rust `lancedb::column_definitions` without loading embedding models. It supports explicit and default Rust vector-column names and respects quoted column paths. Unrelated column changes and nullability updates remain supported. The restriction is documented in the Python, Rust, and TypeScript APIs. Regression coverage includes both SDK metadata formats, native Rust definitions with explicit and default destinations, atomic rejection of batched changes, and continued embedding generation after reopening. Fixes #4763 Validation at the exact PR head and GitHub test-merge: - Hosted macOS Python wheel suite: 3061 passed, 162 skipped, 70 deselected, including all 8 sync and 4 async embedding schema-evolution cases. - Hosted Windows Rust suite: 1900 library tests passed, 8 embedding integration tests passed, and 107 doctests passed, including the native explicit/default destination and binding guard cases. - Hosted Linux Python 3.10 rerun: 3084 passed, 159 skipped, 50 deselected; the previously observed remote-fork timeout passed on the replacement job. - All 10 applicable exact-head workflow runs completed successfully. The existing main-only Windows ARM job was skipped by its workflow condition. |
||
  TaylorandBrendan Clement
|
9e65bd2c27 |
docs: use Enterprise terminology across website and SDK documentation (#4787)
Use “LanceDB Enterprise” throughout the staged website guides, SDK reference sources, generated API pages, and user-facing SDK messages. A separate branded “LanceDB Distributed” name adds terminology without clarifying behavior; describe where operations run directly. - Update `docs/web`, including the architecture diagram title and alt text. Use Enterprise in headings, table labels, and prose; simplify redundant product/execution-model wording and name the product only once in the diagram alt text. - Update Python docstrings and the Python/Java reference pages, TypeScript comments and native declaration comments, and Rust API documentation. Regenerate the TypeScript reference, including 24 replacements across six pages. - Update SDK warnings/errors and existing test expectations to match, along with contributor guidance and Vale rules. Remove the obsolete rule distinguishing Enterprise from Distributed. - Keep lowercase “distributed” for technical descriptions. API identifiers, anchors, asset paths, and execution behavior are unchanged. No branded-name exceptions are proposed. Validation: - Python and Node native builds succeeded; focused Python tests passed (36 passed, 4 skipped), and focused Node tests passed (14). - TypeDoc generation and the complete MkDocs reference build passed. MkDocs retains an existing unresolved `lancedb.functions.SecretReference` cross-reference. - Rust formatting, Node lint/format, changed-file Python lint/format, 32 Vale tests, and whitespace checks passed. Repository-wide Python lint has 48 existing findings in six untouched files. - Combined documentation assembly passed. Mintlify validation and link/anchor/redirect checks previously passed for both site builds; no code snippets changed. Companion PRs: [saved editions, terminology guide, and shared rules](https://github.com/lancedb/docs/pull/413), [Enterprise overview and writing rules](https://github.com/lancedb/sophon/pull/9511). --------- Co-authored-by: Brendan Clement <brendan.clement@lancedb.com> |
||
|
Lance Release
|
558f64c45f | Bump version: 0.41.0-beta.17 → 0.42.0-beta.0 | ||
  LanceDB RobotandJack Ye
|
8893443a68 |
chore: update lance dependency to v15.0.0-beta.8 (#4792)
Update the Rust workspace and Java SDK dependencies to [Lance v15.0.0-beta.8](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.8), preserve the optional `manifest_size` field in remote version responses, and verify directory-namespace renames preserve registration and data both within and across namespaces. Update the Rust and TypeScript API documentation to reflect directory-namespace rename support. Rust callers constructing `Version` literals must now provide `manifest_size` (`None` when unavailable). Java is intentionally pinned to beta.8 even while Maven artifacts are still publishing; dependency-resolution CI failures caused solely by that publishing delay are accepted for this upgrade. Python package pins remain unchanged. --------- Co-authored-by: Jack Ye <yezhaoqin@gmail.com> |
||
 Prince Jain
|
8bc11bfcfc |
test(nodejs): cover camelCase column names across table surfaces
References #3211. The core `col()` camelCase integration coverage requested by #3211 is already on `main` from #4493. This PR adds the adjacent raw-SQL table-surface regressions that were previously uncovered: - unquoted `where("camelCase = 1")` - `update` and `delete` with unquoted camelCase predicates - `mergeInsert` with a camelCase join key, including matched-update and not-matched-insert input - `addColumns` `valueSql` referencing a camelCase column - `createIndex` on a camelCase column These are characterization tests for the existing local-backend behavior. They do not add `col()` coverage or require a live remote service. |
||
|
lancedb-gatefixer[bot]
|
156c12b017 |
test(nodejs): cover local object-form table creation across connections
## Summary - add a regression test for object-form `createTable` on two independent local Node.js connections - verify both connections use the same table name while preserving each database's rows ## Scope This test covers the public local `connect(tmpDir)` path. It does not exercise LanceDB Cloud or remote create requests, and it does not establish a remote root cause or a complete fix for #952. References #952 |
||
  ryan-haynesandRyan Haynes
|
d316cbbed4 |
fix!: support opt-in PKCE for device-code login (#4747)
## Problem and change
Some Keycloak client policies require S256 PKCE during device-code
login. Sending those parameters unconditionally breaks interoperability
with providers that expect standard RFC 8628 requests.
Configure device login with `OAuthFlow::DeviceCode(DeviceCodeOptions)`,
following the existing authorization-code options pattern.
`DeviceCodeOptions::new()` keeps PKCE disabled; `.use_pkce(true)` sends
an S256 challenge during device authorization and its matching verifier
during token polling. There is no automatic downgrade, and enabling PKCE
preserves the existing token-cache identity. Python, Node, and CLI
conversions retain ordinary device login without PKCE.
OIDC discovery and token requests now include a versioned User-Agent.
`env!("CARGO_PKG_VERSION")` is expanded at compile time and embeds the
version in the binary; no Cargo environment variable is needed when
running an installed binary or native binding.
The supported-flows guide documents the Rust-only device PKCE option and
its default.
## Rust API migration
BREAKING CHANGE: `OAuthFlow::DeviceCode` now carries
`DeviceCodeOptions`. Replace construction with
`OAuthFlow::DeviceCode(DeviceCodeOptions::new())` or
`OAuthFlow::DeviceCode(Default::default())`, and match with
`OAuthFlow::DeviceCode(_)`. Import `DeviceCodeOptions` from
`lancedb::remote` to opt into PKCE. Python and TypeScript public APIs
are unchanged.
## Validation
- `cargo +1.97.1 test --locked -p lancedb --features remote --lib
remote::`: 679 passed, including PKCE-on/off polling, option defaults,
User-Agent headers, and cache identity.
- `cargo +1.97.1 test --locked -p lancedb-python -p lancedb-nodejs --lib
test_device_code_conversion`: both native-binding conversion tests
passed.
- `cargo +1.97.1 test --locked -p lancedb --features remote --doc
remote::`: 6 passed, including the new options example and updated
session examples.
- `cargo +1.97.1 test --locked -p lancedb-cli --lib auth::tests::`: 11
passed.
- `cargo +1.97.1 fmt --all -- --check` and `git diff --check`: passed.
- A compiled version-string check ran successfully with
`CARGO_PKG_VERSION` unset at runtime.
---------
Co-authored-by: Ryan Haynes <ryan.haynes@lancedb.com>
|
||
|
Jack Ye
|
ac7930fa77 |
feat: use SQL for remote view lifecycle
Stacked on #4675. Remote view create, describe, list, and drop now execute SQL. CREATE returns the stored definition, resolution defaults, and Arrow schema from the same execution; DESCRIBE VIEW returns that metadata and SHOW VIEWS supports scoped pagination. `drop_view_async()` returns a `job_type="query"` handle whose single `job.id` is the SQL query UUID. Waiting drains the query result once and caches completion; `describe_query(job.id)` works on the submitting connection. Completion means catalog removal, with physical cleanup continuing separately. Query handles have no registry history and cannot be reopened through `open_job()`. |
||
|
Brendan ClementandClaude Opus 5.5
|
ca343a5c06 |
docs: fix table, accuracy and structure problems found in the pre-launch review (#4745)
## Summary Fixes the problems a page-by-page review of the docs found before launch, after checking each one against the current source and, where noted, the code. The matching fixes for the dataset and Geneva pages, which live in `lancedb/docs`, are in lancedb/docs#402. **Tables** - `integrations/embedding/cohere`: each input-type description was split across two table rows and wrapped in literal quotes (`"`search_document`"`). Now one row per type. - `integrations/embedding/gemini`: removes the same stray quotes from the task-type table. - `integrations/ai/langchain`: the MMR search table's `kwargs` row was missing a cell, so its description sat under Type. - Six reranker pages (answerdotai, cohere, colbert, jina, openai, voyageai): closes the backtick before `_relevance_score`. - `indexing/vector-index`: finishes a cell cut off at "with scalar". - `sql/dql/dedupe`: labels the empty header cell in the resolution table. **Corrections** - `curation/sampling`: sampling works in LanceDB Embedded through the Python SDK, so the page is available in both, with the SQL sections marked Enterprise. - `feature-engineering/index`, `add-computed-column`: automatic refresh is on by default for each computed column and takes effect once a deployment enables its scheduler. `refresh-column` already said so; these two implied it was off. - `indexing/scalar-index`, `indexing/index`: `LABEL_LIST` accelerates `array_has_all`/`array_has_any`; `array_contains_*` doesn't exist. Also merges scalar-index's two "nested fields" sections into one. - `integrations/embedding/ibm`: lists the four current models from `CURRENT_MODELS` (it listed a deprecated one) and explains that the default stays a legacy ID so existing tables reload with their model. - `integrations/reranking/openai`: the class is `OpenaiReranker`, not `OpenAI()`. - `integrations/ai/synthetic-data-kit`: the CLI is `synthetic-data-kit` throughout, each step runs one command, each step passes the file it writes to the next, and `save-as` uses `-f`/`-o`. - `integrations/data/dlt`: `secrets.toml` defines `api_token`, which the pipeline reads, and the run command runs the file the page creates. - `search/vector-search`: the multivector walkthrough connected to Enterprise in step 1 and a local path in step 2. It now runs locally throughout, with a note on connecting to Enterprise, and is renamed so it doesn't read as a duplicate of the short section above. **Structure and wording** - `integrations/embedding/nvidia`: adds headings, and drops test-suite details aimed at contributors. - `integrations/ai/genkit`: makes the retrieval test a real heading and describes the query step. - `integrations/data/voxel51`: replaces a raw MkDocs `!!! note` with a `<Note>`. - `sql/ddl/table`, `sql/ddl/job`: "Errors and limits" is a real heading, not bold text. - `curation/windowing`: sidebar title matches the curation overview ("Rank and Deduplicate"). - `deployment/storage`, `basics/agentic-skills`: fixes a garbled sentence and a stale reference to the quickstart. **Checked:** no new Vale alerts on the changed pages. Assembled together with lancedb/docs#402, the site passes `mint validate` and `mint broken-links --check-anchors --check-redirects`. Every changed page was reviewed in a local preview. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
d5013459eb |
chore: update lance dependency to v15.0.0-beta.6 (#4742)
Updates Rust and Java Lance dependencies to [v15.0.0-beta.6](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.6), pins Python tests to published pylance 15.0.0b6, and refreshes both lockfiles. Aligns Arrow 59, DataFusion 55, and PyO3 0.29; adapts execution plans and MemWAL directory handling, updates the ABI metadata test, and removes the obsolete Lance Arrow patch now fixed upstream. Validation: workspace clippy with all features/tests and warnings denied, cargo fmt, Python/CI Ruff checks, uv lock check, 16 CI-script tests, and targeted Python regressions (54 passed, 28 skipped); repository-wide Ruff retains existing documentation-example failures, and the upstream macOS wheel build failed while Linux and Windows wheels are published. --------- Co-authored-by: Daniel Rammer <hamersaw@protonmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
 geoffgscott
|
7ea10fa1e0 |
feat(node): expose targetPartitionSize on IVF index options (#4594)
The Node SDK's vector index options only take `numPartitions`. The Rust index builders (`target_partition_size` in `rust/lancedb/src/index/vector.rs`) and the Python SDK already support a target partition size, so this adds `targetPartitionSize` to `IvfPqOptions`, `IvfRqOptions`, `IvfFlatOptions`, `HnswPqOptions` and `HnswSqOptions` and passes it to their builders. When both are set, `numPartitions` is used, as it is for Rust and Python callers. The reason to want it: an index built from a partition count records no target partition size, so `table.optimize()` splits and joins it against the index type's default and can merge small partitions down to one or two. Building with a target records it in the index details, and optimize keeps to it. https://github.com/lance-format/lance/pull/10154 fixes this on the Lance side for indexes built from a partition count; this lets Node users set the size directly. The new tests build IvfPq, IvfFlat, IvfRq, IvfHnswPq and IvfHnswSq indexes with `targetPartitionSize` and check the size recorded in the index details, and check that `numPartitions` wins when both are set. Each option's doc comment states that index type's default target. Docs regenerated with `pnpm run docs`. |
||
 Will JonesandClaude Opus 5.5
|
4d7a64712f |
docs: adopt Embedded/Distributed terminology across docs and SDKs (#4713)
## Summary Adopts the product terminology guide across the docs site pages, SDK docstrings and error messages. "LanceDB OSS" and local mode become **LanceDB Embedded**. "Remote" as the execution model becomes **LanceDB Distributed**, which LanceDB Enterprise provides. Where free vs paid is the point, pages contrast "LanceDB Embedded (open source)" with "LanceDB Enterprise". This is a first trial of the Embedded name, so reviewers can judge it in context. - **Embedded vs Distributed** for where execution runs (`optimize()`, consistency, branching, maintenance). Main pages define it once: "LanceDB Enterprise runs LanceDB Distributed: …" - **LanceDB Cloud** is kept only for legacy-Cloud specifics (`db://` region, the brute-force KNN limit) and marked legacy. "Cloud and Enterprise" becomes Enterprise or Distributed. - **Plan executor** becomes "executor" or the fleet name. - **Error messages** change, e.g. `optimize is not supported for remote tables.` → `... in LanceDB Distributed.` Tests that match on them are updated. - **Vale rules** (`EmbeddedName`, `OSSProduct`, `RemoteExecution`, `DistributedEnterprise`, `LegacyCloud`, `InternalNames`) enforce the terms in prose. Unchanged: identifiers (`availability.oss`, the `remote` module and feature, `RemoteTable`, `db://`), heading anchors, internal code comments, and Geneva. ## Evidence - Rust lib tests for the changed messages and the Python tests in `test_remote_db.py`, `test_materialized_views.py` and `test_table.py` pass locally. - Not run locally: - The Node tests: `remote.test.ts` and `table.test.ts` have updated assertions. - Vale. - The `experimental_functions`-marked materialized-view tests. - `docs/src` API reference was hand-edited to match the docstrings rather than regenerated. ## Merge Risk **Door:** two-way; prose and message text only. **Blast Radius:** `docs`, `error-messages`: callers matching on old error strings would break. Companion PRs: lancedb/docs (assembler labels, STYLE.md) and sophon (Enterprise pages). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
 JiMyung Lee
|
057a39a1c7 |
docs: row counts in list_indices can be set for remote tables
Remote list_indices fills num_indexed_rows / num_unindexed_rows from the server's enriched response and only leaves them unset for servers that do not report them. The Node and Rust documentation previously implied those fields were always unavailable for remote tables. Synchronize the generated TypeScript API reference with the Node source comments. Closes #4301 Co-authored-by: desty <lee.ji.myung@gmail.com> |
||
 Joshua Snider
|
edb79d38e3 |
docs: correct the documented default retry statuses (#3964)
#2349 changed the default retryable status codes to `[409, 429, 500, 502, 503, 504]`, but only the Rust docstring was updated. The Python and TypeScript bindings — and the generated TypeScript API reference — still document the old `[429, 500, 502, 503]`. The actual default is set in `ResolvedRetryConfig::try_from` in `rust/lancedb/src/remote/retry.rs`, and `rust/lancedb/src/remote/client.rs` already documents it correctly. This brings the other three in line. Docs only — no behavior change. Split out from #3963 per the "one problem at a time" guidance in `CONTRIBUTING.md`. |
||
 J Ganesh Kumar Reddy
|
12d86ea251 |
feat(nodejs): add col() expression builder for camelCase column names
## Summary Fixes #3211 by exposing the shared Rust/DataFusion expression builder through Node bindings. Case-sensitive references such as `col("firstName")` can be composed and passed to `where()`, while SQL strings remain supported. The expression-object representation follows [the design requested on #3290](https://github.com/lancedb/lancedb/pull/3290#discussion_r3632371998). That earlier PR remains open; no claim is made that the issue was unclaimed or that maintainers have selected this implementation. ## Review corrections - Construct the named native struct correctly and parse casts through `arrow_schema::DataType`. - Preserve numeric values: use Int64 only for JavaScript safe integers; retain Float64 otherwise. Explicit `litInt` rejects unsupported numbers with a documented `RangeError`. - Correct test syntax/API calls and align SQL assertions with the shared dialect, including reserved and case-sensitive names. - Use typed, isolated temporary test tables with cleanup, and cover an actual large-number filter threshold. - Document the public methods and commit generated API documentation. Synced with current main `0bceaaf3` through a normal merge. The diff is limited to seven Node source/test/config files and eleven generated documentation files; no Rust core change. ## Validation [Linux validation run](https://github.com/Ganesh1729-ui/lancedb/actions/runs/38244794288) on Node 24 / Rust 1.97.0 with the repository's `ci` profile and `remote,lancedb/metrics-otel` features: - Rebuilt native addon, TypeScript build, Biome, Rust formatting and native Clippy pass. - Expression/query/Arrow/table suites: **1,129 passed**; expression suite repeated: **42 passed**. - Using the original JavaScript numeric dispatch with the rebuilt addon: **8 regressions fail, 2 controls pass**. - Strict TypeDoc generation passes. Final local typing/docs/lint checks pass; the PR source equals the validated source except for the separate validation workflow, which is excluded from this PR. Full provider/model/S3, other feature combinations and OS/Node matrices were not run in this follow-up. Upstream checks and maintainer review remain separate. ## AI assistance AI was used for implementation, review and testing. No prior human review is claimed. |
||
|
Lance Release
|
ece54d8917 | Bump version: 0.41.0-beta.16 → 0.41.0-beta.17 | ||
|
Lance Release
|
45f987a29b | Bump version: 0.41.0-beta.15 → 0.41.0-beta.16 | ||
 Wyatt Alt
|
bb6aa1a147 |
feat: add a cascade option to drop_database (#4697)
The remote catalog could only drop an empty database: it always asked
the server for a restricted drop.
`DropDatabaseRequest::cascade`, `drop_database(..., cascade=True)` in
Python and `dropDatabase(name, { cascade: true })` in TypeScript ask the
server to drop everything in the database first. The name stays taken
until the server finishes cleaning up the dropped objects' storage. A
restricted drop remains the default.
|
||
|
Jack YeandXuanwo
|
45dbf5eff5 |
fix: use SQL for all remote materialized view APIs (#4673)
Remote materialized-view APIs now use SQL for the full lifecycle. Create and refresh already submit SQL; this change moves describe, paginated listing, and drop onto `DESCRIBE MATERIALIZED VIEW`, `SHOW MATERIALIZED VIEWS`, and `DROP MATERIALIZED VIEW`. Async create, refresh, and drop return the existing Job handle with `job_type = "query"`. The handle has one `id`: the connection-scoped SQL query ID, also accepted by `describe_query`. Submission starts the query, the job drains its result once, and repeated waits reuse the completed result. Synchronous drop waits for catalog removal; physical cleanup proceeds in the background. Listing uses namespace-scoped keyset pagination with opaque resume tokens. Streamed and deferred SQL responses share the same path. Adds transport coverage and updates Python/TypeScript documentation for query-backed drop jobs. --------- Co-authored-by: Xuanwo <github@xuanwo.io> |
||
 Dan Tasse
|
6e4f523e14 |
docs: remove logical-column from docs (#4693)
We've removed this feature from LanceDB Enterprise and no longer privilege the "lancedb:logical-column" key. So, removing that from these docs. |
||
 dependabot[bot]
|
d44609b6f2 |
chore(deps): bump the nodejs-deps group across 1 directory with 3 updates (#4661)
Bumps the nodejs-deps group with 3 updates in the /nodejs directory: [@opentelemetry/api](https://github.com/open-telemetry/opentelemetry-js), [@opentelemetry/sdk-metrics](https://github.com/open-telemetry/opentelemetry-js) and [ts-jest](https://github.com/kulshekhar/ts-jest). Updates `@opentelemetry/api` from 1.9.1 to 1.10.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/open-telemetry/opentelemetry-js/releases">@opentelemetry/api's releases</a>.</em></p> <blockquote> <h2>api/v1.10.0</h2> <h2>1.10.0</h2> <h3>🚀 Features</h3> <ul> <li>feat(api): integrate the Logs API into <code>@opentelemetry/api</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/4862">#4862</a> <a href="https://github.com/hectorhdzg"><code>@hectorhdzg</code></a></li> <li>feat(api): add imperative <code>context.attach()</code> for setting context across callback boundaries that <code>with()</code> cannot wrap. <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6845">#6845</a> <a href="https://github.com/pichlermarc"><code>@pichlermarc</code></a> <ul> <li><code>attach()</code> returns a <code>Token</code> whose <code>dispose()</code> method restores the previous context. Use <code>token.dispose()</code> to detach.</li> <li><code>attach</code> is an optional method on the <code>ContextManager</code> interface; when the active context manager does not implement it, <code>context.attach()</code> logs a warning and returns a no-op token.</li> </ul> </li> <li>feat(api): add <code>AnyValue</code> type (of type <code>unknown</code>) and extend <code>Attributes</code> type values to use <code>AnyValue</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6780">#6780</a> <a href="https://github.com/trentm"><code>@trentm</code></a> <ul> <li>This is part of supporting <a href="https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/4485-extending-attributes-to-support-complex-values.md#how">OTEP 4485</a>.</li> <li>This is a <strong>breaking change for SDK implementations</strong>, in that they may need to support more attribute types for any APIs that accept <code>Attributes</code> or <code>AnyValue</code>. Implementors should refer to the OTEP 4485 link above for "MUST / SHOULD" requirements for complex attributes handling. See also the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/packages/opentelemetry-core/src/common/attributes.ts">attributes-handling utilities provided by <code>@opentelemetry/core</code> to help</a>. For example, <code>cleanSimpleAttributes()</code> can be used to limit user-provided attributes to the set of value types supported in earlier <code>@opentelemetry/api</code> versions.</li> <li>This change is <strong>typically not a breaking change for users</strong> of <code>@opentelemetry/api</code>. There is one possible breaking case: If a TypeScript user was using the <code>Attributes</code> type in their own code and using <a href="https://www.typescriptlang.org/docs/handbook/2/narrowing.html">TypeScript Narrowing</a> to reduce an attribute value to an expected type (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6579#issuecomment-4345316200">example here</a>). It is expected that this usage should be rare and, likely, limited to SDK implementations, if at all. (If this impacted your code, <a href="https://github.com/open-telemetry/opentelemetry-js/issues/new">please let us know</a>.)</li> <li><strong>Note for users of attributes:</strong> Simple attributes SHOULD be used whenever possible. Instrumentations SHOULD assume that backends do not index individual properties of complex attributes, that querying or aggregating on such properties is inefficient and complicated, and that reporting complex attributes carries higher performance overhead.</li> </ul> </li> </ul> <h3>🐛 Bug Fixes</h3> <ul> <li>fix(api): cap tracestate list-members and total length in the deprecated <code>TraceState.set()</code> so it stays in step with <code>TraceState</code> from <code>@opentelemetry/core</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6964">#6964</a> <a href="https://github.com/Zuhef"><code>@Zuhef</code></a></li> <li>fix: resolve the ESM build through the <code>default</code> export condition instead of <code>import</code>, so a resolver that activates neither <code>import</code> nor <code>require</code> still finds an entry point <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7132">#7132</a> <a href="https://github.com/overbalance"><code>@overbalance</code></a></li> </ul> <h3>📚 Documentation</h3> <ul> <li>docs(api): complete <code>@since</code> annotations for the public API <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7023">#7023</a> <a href="https://github.com/nabeelamjadsheikh"><code>@nabeelamjadsheikh</code></a> <ul> <li>Adds the three missing exported-type annotations and the member-level annotations for methods and properties introduced after their enclosing type.</li> <li>Corrects <code>Attributes</code>, <code>AttributeValue</code> and <code>TracerOptions</code> from <code>1.3.0</code> to <code>1.1.0</code>; all three were already importable in 1.1.0 via wildcard re-export.</li> </ul> </li> </ul> <h3>🏠 Internal</h3> <ul> <li>build: migrate the package build from <code>tsc</code> to <code>tsdown</code>, emitting dual CJS/ESM output from a single <code>dist/</code> directory <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6293">#6293</a> <a href="https://github.com/overbalance"><code>@overbalance</code></a> <ul> <li>The public entry points are unchanged: <code>@opentelemetry/api</code> and <code>@opentelemetry/api/experimental</code>. This package already declared an <code>exports</code> map, so deep imports such as <code>@opentelemetry/api/build/src/...</code> were rejected before this change and continue to be.</li> <li>The emitted files moved out of <code>build/src</code> (CJS), <code>build/esm</code> and <code>build/esnext</code> (ESM) into <code>dist/</code>, using <code>.cjs</code>/<code>.mjs</code> extensions with matching <code>.d.cts</code>/<code>.d.mts</code> declarations. The non-standard <code>esnext</code> condition has been dropped in favour of the standard <code>import</code>/<code>require</code> conditions.</li> </ul> </li> <li>perf(api): add getGlobal fast-path <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6956">#6956</a> <a href="https://github.com/legendecas"><code>@legendecas</code></a></li> <li>chore: remove the redundant <code>module</code> field from <code>package.json</code>; bundlers resolve the ESM build through the <code>exports</code> map <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7130">#7130</a> <a href="https://github.com/overbalance"><code>@overbalance</code></a></li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md">@opentelemetry/api's changelog</a>.</em></p> <blockquote> <h2>1.10.0</h2> <h3>🚀 (Enhancement)</h3> <ul> <li>feat(resource): create sync resource with some attributes that resolve asynchronously <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3460">#3460</a> <a href="https://github.com/samimusallam"><code>@samimusallam</code></a></li> <li>feat: collect host id for non-cloud environments <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3575">#3575</a> <a href="https://github.com/mwear"><code>@mwear</code></a></li> <li>feat (api-logs): separate Events API into its own package <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3550">3550</a> <a href="https://github.com/martinkuba"><code>@martinkuba</code></a></li> <li>feat(sdk-metrics): apply binary search in histogram recording <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3539">#3539</a> <a href="https://github.com/legendecas"><code>@legendecas</code></a></li> <li>perf(propagator-jaeger): improve deserializeSpanContext performance <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3541">#3541</a> <a href="https://github.com/doochik"><code>@doochik</code></a></li> <li>feat: support TraceState in SamplingResult <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3530">#3530</a> <a href="https://github.com/raphael-theriault-swi"><code>@raphael-theriault-swi</code></a></li> <li>feat(sdk-trace-base): add diagnostic logging when spans are dropped <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3610">#3610</a> <a href="https://github.com/neoeinstein"><code>@neoeinstein</code></a></li> <li>feat: add unit to view instrument selection criteria <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3647">#3647</a> <a href="https://github.com/jlabatut"><code>@jlabatut</code></a></li> <li>feat(tracing): expose dropped counts for attributes, events and links on span <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3576">#3576</a> <a href="https://github.com/mohitk05"><code>@mohitk05</code></a></li> </ul> <h3>🐛 (Bug Fix)</h3> <ul> <li>fix(core): added falsy check to make otel core work with browser where webpack config had process as false or null <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3613">#3613</a> <a href="https://github.com/ravindra-dyte"><code>@ravindra-dyte</code></a></li> <li>fix(instrumentation-http): include query params in http.target <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3646">#3646</a> <a href="https://github.com/kobi-co"><code>@kobi-co</code></a></li> </ul> <h3>📚 (Refine Doc)</h3> <ul> <li>chore: update http example <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3651">#3651</a> <a href="https://github.com/JamieDanielson"><code>@JamieDanielson</code></a></li> </ul> <h3>🏠 (Internal)</h3> <ul> <li>chore(exporter-jaeger): deprecate jaeger exporter <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3585">#3585</a> <a href="https://github.com/pichlermarc"><code>@pichlermarc</code></a></li> <li>fix(sdk-metrics): fix flaky LastValueAggregator test by using fake timer <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3587">#3587</a> <a href="https://github.com/pichlermarc"><code>@pichlermarc</code></a></li> <li>fix(test): fix failing tests by preventing source-map generation <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/3642">#3642</a> <a href="https://github.com/pichlermarc"><code>@pichlermarc</code></a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/56e6b1bb890f844b8963a146780d0b9cfa8abd0d"><code>56e6b1b</code></a> chore: Release API 1.4.1, SDK 1.10.0, Experimental 0.36.0 (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3654">#3654</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/ecb5ebe86eebc5a1880041b2523adf5f6d022282"><code>ecb5ebe</code></a> fix: Minor typing issues for web sandbox (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3660">#3660</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/48ecf229eb0b468cf433a152daf414313b381edc"><code>48ecf22</code></a> feat(sdk-trace-base): add dropped attributes and events count on span (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3576">#3576</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/95bf6fc7cc105469def055f1f93446c9cd09882d"><code>95bf6fc</code></a> fix(build): add <code>example/http</code> to ignore list in <code>update-ts-configs</code> (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3657">#3657</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/3a2ca58075cc7e0eea8233c51735db5d583c11d0"><code>3a2ca58</code></a> feat: add unit to view instrument selection criteria (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3647">#3647</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/ebc8575a0c215a7b7c84866116a261ffd8aff015"><code>ebc8575</code></a> chore(jeager-exporter): deprecate jaeger exporter (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3585">#3585</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/2f715bd78dd7fb66e2aa2b76fb6e7c623deddd60"><code>2f715bd</code></a> feat: allow to add attributes from grpc metadata in the patched server (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3589">#3589</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/cbbdfd379854fcb41bd70841ae4890200a2fda09"><code>cbbdfd3</code></a> fix(sdk-node): update instrumentations once MeterProvider is initialized (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3624">#3624</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/4a4484a910201466121ed45fa12566737f6ac19d"><code>4a4484a</code></a> chore: update http example (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3651">#3651</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/494efffcf4648e3b1207c3c925c1a52bcc7f7350"><code>494efff</code></a> fix(otlp-exporter-base): fix changelog for merged retry pr (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/3650">#3650</a>)</li> <li>Additional commits viewable in <a href="https://github.com/open-telemetry/opentelemetry-js/compare/v1.9.1...v1.10.0">compare view</a></li> </ul> </details> <br /> Updates `@opentelemetry/sdk-metrics` from 2.11.0 to 2.12.0 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/open-telemetry/opentelemetry-js/releases">@opentelemetry/sdk-metrics's releases</a>.</em></p> <blockquote> <h2>v2.12.0</h2> <h2>2.12.0</h2> <h3>🚀 Features</h3> <ul> <li>chore(sdk-trace): add <code>StackContextManager</code> in <code>@opentelemetry/sdk-trace</code> package <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7086">#7086</a></li> </ul> <h3>🏠 Internal</h3> <ul> <li>feat(ci): support releasing from maintenance branches <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6767">#6767</a> <a href="https://github.com/pichlermarc"><code>@pichlermarc</code></a> <ul> <li>The API documentation site is only redeployed for releases whose commit is reachable from <code>main</code>, so a maintenance release no longer overwrites it.</li> </ul> </li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md">@opentelemetry/sdk-metrics's changelog</a>.</em></p> <blockquote> <!-- raw HTML omitted --> <h1>CHANGELOG</h1> <p>All notable changes to this project will be documented in this file.</p> <p>For API changes, see the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/api/CHANGELOG.md">API CHANGELOG</a>. For experimental package changes, see the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/experimental/CHANGELOG.md">experimental CHANGELOG</a>. For semantic convention package changes, see the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/semantic-conventions/CHANGELOG.md">semconv CHANGELOG</a>. For notes on migrating to 2.x / 0.200.x see <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/upgrade-to-2.x.md">the upgrade guide</a>. For notes on migrating to 3.x see <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/3.x/migration-guide.md">the 3.x migration guide</a>.</p> <h2>Unreleased</h2> <h3>💥 Breaking Changes</h3> <h3>🚀 Features</h3> <h3>🐛 Bug Fixes</h3> <h3>📚 Documentation</h3> <h3>🏠 Internal</h3> <h2>3.0.0</h2> <h3>💥 Breaking Changes</h3> <ul> <li>chore(sdk-trace-node)!: remove the <code>@opentelemetry/sdk-trace-node</code> package <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7054">#7054</a> <ul> <li>The sdk-trace-node package has been replaced by the <code>@opentelemetry/sdk-trace</code> package. See the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/3.x/migration-guide.md">3.x migration guide</a> for full migration instructions.</li> </ul> </li> <li>chore(shim-opentracing)!: remove the <code>@opentelemetry/shim-opentracing</code> package <ul> <li>In the <a href="https://github.com/open-telemetry/opentelemetry-specification/releases/tag/v1.56.0">OpenTelemetry Specification v1.56.0</a> the <a href="https://redirect.github.com/open-telemetry/opentelemetry-specification/pull/4938">OpenTracing compatibility requirements were deprecated</a>. The JavaScript OpenTracing shim package will not receive any more releases after the current <a href="https://www.npmjs.com/package/@opentelemetry/shim-opentracing/v/2.12.0">2.12.0 release</a> (<a href="https://github.com/open-telemetry/opentelemetry-js/tree/v2.12.0/packages/opentelemetry-shim-opentracing/">source code for last release</a>).</li> </ul> </li> <li>chore(context-async-hooks)!: remove the unused class <code>AsyncHooksContextManager</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7078">#7078</a></li> <li>feat!: migrate package builds from <code>tsc</code> to <code>tsdown</code>, emitting dual CJS/ESM output from a single <code>dist/</code> directory and declaring an <code>exports</code> map on every package <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6293">#6293</a> <a href="https://github.com/overbalance"><code>@overbalance</code></a> <ul> <li>Importing a package by its name is unaffected in both CommonJS and ESM, as is every subpath listed in its <code>exports</code> map.</li> <li><strong>Deep imports into the build output no longer resolve.</strong> An <code>exports</code> map is an allowlist that Node.js and bundlers enforce, so specifiers such as <code>@opentelemetry/core/build/src/...</code> or <code>@opentelemetry/core/build/esm/...</code> now fail with <code>ERR_PACKAGE_PATH_NOT_EXPORTED</code>. Rewriting them to the new file layout does not help — unlisted subpaths are rejected whether or not the file exists.</li> <li>The emitted files moved out of <code>build/src</code> (CJS), <code>build/esm</code> and <code>build/esnext</code> (ESM) into <code>dist/</code>, using <code>.cjs</code>/<code>.mjs</code> extensions with matching <code>.d.cts</code>/<code>.d.mts</code> declarations.</li> <li><code><package>/package.json</code> is no longer importable for the same reason. Read a package's version from your own dependency metadata, or use <code>SDK_INFO</code> from <code>@opentelemetry/core</code> for the SDK version.</li> <li>The non-standard <code>esnext</code> entry point has been removed; tools that preferred it fall back to <code>module</code> (ESM) or <code>main</code> (CJS).</li> <li>If you depend on something that is only reachable through a deep import, please open an issue so it can be considered for the public API.</li> </ul> </li> <li>feat(core)!: remove deprecated <code>getTimeOrigin</code>, <code>otperformance</code>, <code>_globalThis</code>, and <code>unrefTimer</code> from <code>@opentelemetry/core</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7053">#7053</a> <ul> <li><code>getTimeOrigin()</code> — use <code>performance.timeOrigin</code> directly.</li> <li><code>otperformance</code> — use the global <code>performance</code> object directly.</li> <li><code>_globalThis</code> — use <code>globalThis</code> directly.</li> <li><code>unrefTimer(timer)</code> — call <code>timer.unref()</code> directly in your own code.</li> </ul> </li> <li>feat(sdk-trace)!: remove deprecated <code>TracerProviderOptions.forceFlushTimeoutMillis</code> <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7057">#7057</a> <ul> <li>Pass <code>timeoutMillis</code> to <code>provider.forceFlush({ timeoutMillis })</code> instead. The default timeout is 30000ms.</li> </ul> </li> <li>feat(propagator-jaeger)!: remove <code>@opentelemetry/propagator-jaeger</code> package <a href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7077">#7077</a> <ul> <li>The Jaeger propagator is deprecated by the OpenTelemetry specification in favour of <code>W3CTraceContextPropagator</code>. Use <code>W3CTraceContextPropagator</code> from <code>@opentelemetry/core</code> instead.</li> <li>See the <a href="https://github.com/open-telemetry/opentelemetry-js/blob/main/doc/3.x/migration-guide.md">3.x migration guide</a> for full instructions.</li> </ul> </li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/92b844e8aa7d9655e4b3560154e912c4c1ced64d"><code>92b844e</code></a> chore: prepare next v2.x release (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7151">#7151</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/a859aff9825ec8138cf287d08ac86401f00aad7c"><code>a859aff</code></a> fix(web-common): export getElementXPath utility (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7124">#7124</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/2bd5a73e7af7ab363f3b239f8cb1f5f56eaa3ad5"><code>2bd5a73</code></a> chore(web-common)!: deprecate <code>addSpanNetworkEvents()</code> et al instrumentation-...</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/d60f05edeac6aec8e64736c10672077fd4a766b5"><code>d60f05e</code></a> doc(sdk-trace): add description for StackContextManager (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7089">#7089</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/5bd476e6f7a22e6cf2e94a645833247284aa295d"><code>5bd476e</code></a> chore: move context manager to sdk-trace (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7086">#7086</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/554c0c6467c77e9ef1f2676f2bd20b505e228417"><code>554c0c6</code></a> chore: move sdk-trace-web utils to web-common (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7058">#7058</a>)</li> <li><a href="https://github.com/open-telemetry/opentelemetry-js/commit/788922c5780a17d5aaace76a0da458858b2ad88a"><code>788922c</code></a> ci: support maintenance branch releases (<a href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7036">#7036</a>)</li> <li>See full diff in <a href="https://github.com/open-telemetry/opentelemetry-js/compare/v2.11.0...v2.12.0">compare view</a></li> </ul> </details> <br /> Updates `ts-jest` from 29.4.12 to 29.4.15 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/kulshekhar/ts-jest/releases">ts-jest's releases</a>.</em></p> <blockquote> <h2>v29.4.15</h2> <p>Please refer to <a href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a> for details.</p> <h2>v29.4.14</h2> <p>Please refer to <a href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a> for details.</p> <h2>v29.4.13</h2> <p>Please refer to <a href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a> for details.</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">ts-jest's changelog</a>.</em></p> <blockquote> <h2><a href="https://github.com/kulshekhar/ts-jest/compare/v29.4.14...v29.4.15">29.4.15</a> (2026-10-15)</h2> <h3>Bug Fixes</h3> <ul> <li><strong>compiler:</strong> resolve modules with tsconfig options regardless of cache state (<a href="https://github.com/kulshekhar/ts-jest/commit/268746c0fcaf72476cebb348c5543e5c2aac3622">268746c</a>), closes <a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5216">#5216</a></li> <li>support <code>.cjs</code> config files in <code>config:migrate</code> (<a href="https://github.com/kulshekhar/ts-jest/commit/4001b4e6d28aa461e7468f09157413e113f0bd92">4001b4e</a>), closes <a href="https://redirect.github.com/kulshekhar/ts-jest/issues/4485">#4485</a></li> </ul> <h2><a href="https://github.com/kulshekhar/ts-jest/compare/v29.4.13...v29.4.14">29.4.14</a> (2026-09-25)</h2> <h3>Reverts</h3> <ul> <li>bring back prior changes of compiler util (<a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5471">#5471</a>) (<a href="https://github.com/kulshekhar/ts-jest/commit/01e5a2e17ac9c940880865d6a81751832dcc4184">01e5a2e</a>), closes <a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5469">#5469</a></li> </ul> <h2><a href="https://github.com/kulshekhar/ts-jest/compare/v29.4.12...v29.4.13">29.4.13</a> (2026-09-22)</h2> <h3>Features</h3> <ul> <li>support Babel 8 (<a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5466">#5466</a>) (<a href="https://github.com/kulshekhar/ts-jest/commit/c3a0b57c00d8b6acfb72b020e2411752d39073e2">c3a0b57</a>), closes <a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5412">#5412</a></li> </ul> <h3>Performance Improvements</h3> <ul> <li><strong>compiler:</strong> cache module resolution modes (<a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5419">#5419</a>) (<a href="https://github.com/kulshekhar/ts-jest/commit/697d337c1b79a0804972a3d30c0875b02e73bf62">697d337</a>), closes <a href="https://redirect.github.com/kulshekhar/ts-jest/issues/5402">#5402</a></li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/kulshekhar/ts-jest/commit/197d4de96002589658d8a6c2d3b9b9554d17af6b"><code>197d4de</code></a> chore(release): 29.4.15</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/ebad800d2c080cb72d7f0ea3225221b0cee066da"><code>ebad800</code></a> build(deps): update dependency node to v24.22.0</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/e5f6e986fe7c9e8115a7afe49ec9d5e8a576e93b"><code>e5f6e98</code></a> test(compiler): clean up temp dirs from module resolution tests</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/77dbb184d5895f8e315d574971fc83ab6a8a43b2"><code>77dbb18</code></a> test(compiler): make module resolution tests path-independent on Windows</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/268746c0fcaf72476cebb348c5543e5c2aac3622"><code>268746c</code></a> fix(compiler): resolve modules with tsconfig options regardless of cache state</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/15362016c490ae230c3d69f7a9399513d3600ed7"><code>1536201</code></a> build(deps): update dependency vite to ^8.3.4</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/141e39c092759ca6173095fe2dd4ebcdd3117e62"><code>141e39c</code></a> build(deps): update dependency <code>@babel/core</code> to ^8.0.7</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/6a6686e5374525db09ac67f37cf0f2d53c372f28"><code>6a6686e</code></a> build(deps-dev): bump shell-quote from 1.10.0 to 1.12.0 in /examples</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/98e6e50fe934e8e86c903719048891a3a071a2b7"><code>98e6e50</code></a> build(deps): bump proxy-addr from 2.0.7 to 2.0.8 in /website</li> <li><a href="https://github.com/kulshekhar/ts-jest/commit/d2439ff5ed9f6425bed60d0202f37771022f0526"><code>d2439ff</code></a> build(deps): bump compression from 1.8.1 to 1.8.2 in /website</li> <li>Additional commits viewable in <a href="https://github.com/kulshekhar/ts-jest/compare/v29.4.12...v29.4.15">compare view</a></li> </ul> </details> <br /> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
|
0cccc9c991 |
chore(deps): bump rust-minor-patch group and napi 3.15 (#4669)
Bumps the rust-minor-patch group: serde_json 1.0.154, tokio 1.54.1, uuid 1.28.0, reqwest 0.13.5, webbrowser 1.2.5, napi-derive 3.6.13, napi-build 2.6.0, arc-swap 1.9.3, libc 0.2.191. On top of the Dependabot update: * napi is bumped to 3.15.0 instead of 3.13.0. napi-derive 3.6.13 generates code that references `bindgen_prelude::NativeBorrowScopeRelease`, which napi 3.13.0 doesn't have, so the nodejs crate failed to compile. The napi and napi-derive floors in `nodejs/Cargo.toml` are raised so they can't drift apart again. * The RUSTSEC-2026-0410 ignore from #4683 is removed from `deny.toml`. #4690 already moved the lockfile to zstd-sys 2.1.2, which fixes it. The other three zstd ignores stay until we can move to Arrow 59+. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Will Jones <willjones127@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
lancedb-gatefixer[bot]andXuanwo
|
2cd635d41c |
fix: correct remote materialized view refresh results and options (#4672)
Remote materialized-view refreshes drained the SQL result and then fabricated `mode=rebuild`, `rows_written=<total view rows>`, and `source_version=0`. This made incremental refreshes report incorrect statistics. Make these three result fields optional across Rust, Python, and TypeScript, and leave them unset for remote SQL refreshes. Remove the post-refresh row count request. Local refreshes continue to populate all metrics, and `version` remains the view version observed after completion. Remote per-operation statistics remain unavailable until the SQL job adapter can retrieve them. Expose the forced-rebuild option through the Rust builder, synchronous/asynchronous Python refresh and job methods, and TypeScript refresh. Remote `full=true` sends `WITH (full = true)`; the default lets the server choose its refresh mode. Local source-version pinning and remote job cancellation are preserved. Local refresh regression tests respect the `experimental-functions` build gate, while remote refresh coverage also runs in default builds. Validation: - Rust: 20 materialized-view tests and 21 remote SQL tests passed with `remote,experimental-functions`; `cargo fmt --all` passed. - Python: rebuilt the extension in the uv-managed environment. The default build passed 23 tests with 27 feature-dependent skips. The opt-in build passed 61 tests with 3 expected default-build-only skips, covering the materialized-view suite and 14 async query-serialization cases from current main. Python Ruff lint and formatting checks passed. - TypeScript: default and opt-in native builds and TypeScript compilation passed. The default build passed 5 tests with 10 expected feature-dependent skips; the opt-in build passed all 14 materialized-view tests with the default-build-only test skipped. Lint/format checks and documentation generation passed. - Repository-wide `ruff check .` reports 48 pre-existing violations in unchanged `docs/web-tests` files; `ruff check python` passes. Fixes #4670 <!-- lance-gatekeeper-fix:v1 agent=ece43ec4b88a48d1789f60005fa3ecfc generation=1 --> Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
lancedb-gatefixer[bot]andXuanwo
|
83b7275137 |
fix(remote): make HTTP timeout errors actionable (#4650)
Remote `create_table` reader uploads can exceed the read deadline while waiting for response headers, but reqwest's main error message only says "error sending request". Make HTTP and retry errors explicitly identify timeouts and name the relevant `client_config.timeout_config` settings and environment variables, preserving typed sources, request metadata, and retry classification. Document that the read timeout covers waiting for headers during upload. This implements the issue's diagnostic option: creation still uses one streaming request, so large or slow readers may require a longer `client_config.timeout_config.read_timeout`. Validation: - 48 focused Rust client, error, and retry tests passed. Regression coverage holds an upload open until its read timeout and also exercises TLS connection timeouts, with and without retries. - 8 Python create-table and HTTP/retry error tests passed in the uv-managed environment after rebuilding the extension, including reader creation with read and overall timeouts. - `cargo check --quiet --features remote --tests --examples` passed. - `cargo clippy --quiet --features remote --tests --examples` passed. - Rust formatting and Python package Ruff lint/format checks passed. - Repository-wide `ruff check .` reports 49 existing findings under `docs/web-tests`; those files are unchanged. Fixes #4646 <!-- lance-gatekeeper-fix:v1 agent=ce1c040fa581f1748166ecb686d9c9c0 generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
Xuanwo
|
8a4effa06a |
fix: gate experimental local functions behind an opt-in feature (#4688)
Published SDK builds currently expose local SQL computed columns and materialized-view execution that are not ready for release. Add the opt-in `experimental-functions` Cargo feature to the Rust core and forward it through the Python and Node bindings; published packages leave it disabled. Local declarations and refreshes now return `NotSupported` before mutating tables or spawning jobs, including prepared materialized-view creation. Remote Cloud/Enterprise client operations, Python UDF packaging, ordinary eager `add_columns`, and access to existing data remain available. Stored metadata and write protections stay active, and shared server commit helpers remain usable. CI covers both feature configurations. Existing local execution tests run with the feature enabled; default-build regressions cover early rejection, unchanged table versions, existing metadata protections, and normal eager transforms. API and source-build documentation explain the opt-in. Validation: Rust library tests pass with the feature disabled (1,869 tests) and enabled (1,947 tests), plus three default-build integration regressions. Python and TypeScript bindings were rebuilt and tested in both configurations, including mocked remote requests; Rust formatting, Python package lint, TypeScript lint/build, and generated docs were checked. Repository-wide Ruff still reports 48 pre-existing errors in unchanged `docs/web-tests` files. |
||
|
Lance Release
|
37b46f6b4f | Bump version: 0.41.0-beta.14 → 0.41.0-beta.15 | ||
 KarthikSandXuanwo
|
180eaaf68e |
fix!: reject unsupported remote connection and table options (#4486)
Fixes #4399. Remote connections currently discard local-only settings, including table write options that determine storage format. The synchronous Python client also binds the fourth positional `create_table` argument to `on_bad_vectors`, so an intended overwrite is sent as a create. This change aligns the remote Python signature with `DBConnection.create_table`, preserves the legacy remote timeout keywords, and raises `ValueError` for other unknown connection keywords. The shared Rust client rejects unsupported connection, create, and open options before sending HTTP requests; open validation also runs before a table-cache hit. Python reports these as `NotImplementedError`, including explicit `enable_v2_manifest_paths=False`. Supported Enterprise connection-level storage options remain available. This implements option 2 from the issue: explicit rejection. It deliberately does not add unverified server write parameters. Calls that previously supplied ignored options must omit them to use server-managed storage and caching. Database URI prefixes are already preserved on current main and are unchanged here. Regression tests cover Python sync/async, TypeScript create/open calls, and the Rust remote client, including empty-table creation, overwrite mode, and cached table handles. Python API documentation describes the behavior and corrected positional order. Reconciled with main at `0ea932da`, preserving upstream blob and error-handling tests and updating the Python test that previously expected open options to be ignored. Validation on the merged source: - Rebuilt the Python native extension; 304 tests passed across remote options, database, remote database, and session suites. - Rust remote database suite: 135 passed. Rebuilt the Node native extension; TypeScript remote suite: 62 passed. TypeScript compilation and documentation generation passed. - Full Python SDK lint and formatting pass (159 files); full TypeScript lint and formatting pass (66 files). Rust formatting and `git diff --check` pass. - Repository-wide Python lint/format reports existing issues in unchanged `docs/web-tests` files (49 lint findings and 11 formatting files). Environment: Ubuntu under WSL2, Linux x86-64, CPython 3.12.13, Rust 1.97.0, and Node.js 22.16.0. Locked dependencies and the repository `ci` build profile. Remote tests use HTTP fixtures; this does not claim validation against the original ARM64 single-node deployment. Documentation review follow-up: added a draft Enterprise migration note in `docs/web/basics/sdk.mdx`. It explains that `db://` connections reject local session/cache and per-table storage/write options, while supported connection-level Enterprise storage options remain available. Following the repository's release conventions, the note stays in an MDX comment below the migration list until the behavior is released. Vale 3.24.0 with the pinned LanceDB style package reports no findings; `git diff --check` passes. This follow-up changes documentation only. BREAKING CHANGE: Remote connections now reject unsupported local session/cache and per-table storage/write options instead of silently ignoring them. Callers must omit these options and use server-managed storage and caching; supported Enterprise connection-level storage options remain available. Integration with current main preserves its synchronous API signatures, including the remote `create_table(mode=None)` default, and the table compatibility defaults. The positional-order repair is already on main via #4608; this PR retains the distinct unsupported-option rejection behavior. CI follow-up: the inherited IVF search-effort test occasionally measured recall 0.375 against its unchanged 0.5 threshold after random index training. A matching local reproducer observed this once in 30 builds. The fixture now places the eight exact neighbors together, preserving the 16-partition setup, two writes, all probe/precedence assertions, and the recall threshold. The adjusted fixture returned recall 1.0 in 100 builds; the full query and remote suites passed (102 tests). This stabilizes test data and does not change search behavior or promise production recall. --------- Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
lancedb-gatefixer[bot]andXuanwo
|
66357275bb |
fix: inherit Lance defaults for IVF RQ quantization (#4585)
`IvfRq()` forced `num_bits=1` through Python's Rust binding, overriding Lance's five-bit default. The shared native index creation path also substituted one bit when the Rust or Node builder left the value unset. Make Python's bit count optional and forward only explicit values. Use `RQBuildParams::default()` for an unset native bit count, and omit unset values when dispatching to pylance for accelerator builds. Update the Python, Rust, Node, and vector-index documentation. New indexes with an unset bit count now inherit Lance's default, currently five bits, across Python, Rust, and Node. Five-bit vector codes use roughly five times the storage of one-bit codes, plus other index overhead. This also changes reader compatibility: indexes created with `num_bits >= 2` use a newer on-disk layout that some older LanceDB versions cannot read. Upgrade readers to versions that support multi-bit IVF_RQ, or explicitly set `num_bits=1` (for example, `IvfRq(num_bits=1)` in Python) when sharing indexes with readers limited to one-bit IVF_RQ. Validation: - Reproduced six regression failures before the fix: one-bit default metadata, rejected `None`, and incorrect remote/accelerator forwarding. - After merging `main` through `b762ec22`, rebuilt both native bindings against Lance 15.0.0-beta.1. Python validation passed with 106 tests and 15 optional Ollama skips. Coverage includes the full index suite, omitted and explicit bit counts, reading Lance-created IVF_RQ indexes, accelerator forwarding, CUDA-unavailability handling, timeout validation, remote requests, and table/namespace listing behavior. - Rust formatting and Python package Ruff lint/format checks pass. Node `pnpm lint`, `pnpm build` (including TypeScript checks), `pnpm run docs`, the full connection suite, and the existing IVF_RQ regression pass. Generated documentation adds no further changes. The merge preserves upstream's explicit RaBitQ heading anchor alongside the `IvfRq()` example and five-bit default/reader-compatibility guidance. Repository-wide `ruff check .` still reports 49 existing errors in unchanged documentation examples under `docs/web-tests`; root formatting also finds 11 unchanged documentation example files. The Python package checks pass. Fixes #4582 <!-- lance-gatekeeper-fix:v1 agent=6ef9406f0142b1ab66d53d9d56098223 generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
|
Lance Release
|
de0c5edc7d | Bump version: 0.41.0-beta.13 → 0.41.0-beta.14 | ||
|
Jack Ye
|
9af7757053 |
fix: execute remote materialized view creation through SQL (#4665)
Remote materialized-view creation currently expects a REST response containing a registry job ID, while refresh already submits SQL. Route both through the SQL query backend and preserve Python's synchronous and asynchronous return types. ### Query to Job mapping `SqlClient::submit_as_job()` submits an `ExecuteQueryRequest` through the same backend as `execute_query_async()`. Once submission succeeds, it retains the returned `Query` and starts one client-side task that consumes its result reader to completion. `Job::from_query_task()` wraps that task, its cached outcome, a single ID and a cancellation callback. Python exposes this as `Job[T]` or `AsyncJob[T]`. The existing `job_type` property maps SQL-backed handles to the new `"query"` kind. The shared `id` contains that query's UUID string, so Python callers can inspect it with `db.describe_query(job.id)` on the submitting connection. No separate `query_id`, backend enum or `type()` API is added. | Handle | `job_type` | `id` | Available lookup | | --- | --- | --- | --- | | SQL-backed | `"query"`, known at submission | Connection-scoped query UUID string | `describe_query(job.id)` | | Registry-backed | Existing server-reported kind, after `status()` or `refresh()` | Registry job ID string | Registry job APIs, descriptions and events | | Local | `None` | `None` | The current in-process handle | | Job concept | SQL-backed behavior | | --- | --- | | Submission | Await initial SQL submission before returning the handle. Submission errors propagate immediately. | | `status()` | Tracks the adapter task: running during result consumption/finalization, finished after both succeed, failed if either fails, or cancelled after cancellation. | | `wait()` | Returns the operation's typed terminal result. One task consumes the query reader once; repeated waits reuse its cached outcome without resubmitting SQL or reopening the reader. | | `cancel()` | Delegates to `Query.cancel()`. Rejected or unsupported cancellation leaves the task running; an accepted request stops the task. Cancelling a finished operation is a no-op. | | Registry details | Query-backed handles expose no registry description, event history, pause/resume or reattachment after a client restart. `refresh()` updates cached lifecycle state. | The Job tracks command completion rather than exposing SQL batches. A wait timeout does not cancel the query. Query IDs remain scoped to the submitting connection and cannot be used with registry APIs or as durable resume tokens. Python `describe_query()` accepts both UUID objects and the string `id` exposed by query-backed jobs. Rust, Python and TypeScript expose the same classification through their existing job-type accessors. ### Materialized-view operations - **CREATE:** Render `CREATE MATERIALIZED VIEW` with the view namespace and optional auto-refresh settings. After command completion, resolve the job to `None`. Synchronous creation waits on the job, then opens and returns the materialized view. - **REFRESH:** Submit `REFRESH MATERIALIZED VIEW` through the same adapter. After command completion, read the refreshed table's version and row count to construct the existing typed refresh result. Synchronous refresh waits on the same path. Remote creation requires the SQL service. Execution placement remains a server decision. REST endpoints are unchanged. The job guide documents the ID/job-type contract and query-backed limitations. Regression coverage exercises SQL submission, namespaces, refresh options, job-type/ID preservation through typed result mapping, repeated waits, failures and cancellation rejection. No local builds or tests were run. |
||
 Colin Patrick McCabe
|
631ea41a63 |
feat!: return lazy iterators for table and namespace listings (#4599)
Make table and namespace listings consistent with other resource
listings across Rust, Python, and TypeScript. Callers can iterate over
individual names across all pages, with configurable page sizes and
continuation tokens.
In order to avoid breaking existing Python users, this change adds some
compatibility methods to the returned Python iterators. Code such as:
```
for t in db.list_tables().tables:
print(t)
```
will continue to work as expected because the iterator has a `tables`
field and a `page_token` field, which function as expected to reveal the
first page of results. In other words, it is never necessary to use the
iterator __as an iterator__. It may be used as if it were an old-style
response object. (As the iterator progresses, the `tables` list may
switch to the second, third, etc. page of results.) Similar
compatibility code allows the async python API to keep working as well.
|
||
|
lancedb-gatefixer[bot]andXuanwo
|
f5f4463b15 |
fix(node): materialize nested struct blobs in bytes mode (#4611)
`blobMode: "bytes"` left blobs inside structs as descriptors because projection matching ignored parent structs and byte replacement only rebuilt top-level fields. Expand direct struct projections and aliases to their blob leaves, then recursively replace the corresponding Arrow fields and data with `LargeBinary`. Track literal output names separately from descendant paths so scalar aliases stay intact and every overlapping blob projection is converted. Preserve null structs, null and empty blobs, sibling fields, schema metadata, and the existing row ID and snapshot behavior. Blobs inside lists retain their descriptors. Update the API docs to describe struct support. Regression coverage includes full and parent-struct projections, leaf and struct aliases, dotted output names, overlapping projections in both orders, nested structs, nulls, empty blobs and results, multiple batches, list siblings, query reuse, row IDs, and concurrent writes. Regression tests reproduce the nested-blob and projection-collision failures before the corresponding changes. Validation: - `pnpm build` - `pnpm lint` - `pnpm run docs` - `pnpm test __test__/nested_blob.test.ts __test__/blob.test.ts __test__/query.test.ts --runInBand --verbose=false` — 142 passed - `pnpm test __test__/table.test.ts --runInBand --verbose=false -t 'when dealing with blob columns'` — 60 passed Fixes #4609 <!-- lance-gatekeeper-fix:v1 agent=76edaac77d044c468d569b58efa2104a generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io> |
||
 Ishita Singh
|
e5873d9763 |
fix(node): support nested blobs with blobMode 'bytes' (#4610)
Closes #4609. `blobMode: "bytes"` only resolved top-level blob columns, so a blob nested in a struct came back as its descriptor. Now struct projections that contain blob columns are resolved too: the bytes are fetched per blob column and the struct is rebuilt with `LargeBinary` leaves, preserving struct validity so a null struct stays null. Sibling fields pass through untouched, aliases work for both `info.image AS image` and `info AS details`, and blobs nested in lists still keep their descriptors. Tested with `npx tsc`, `biome check`, and the full `table.test.ts` suite (580 tests, including new regressions for select-all, struct selection, aliases, null structs, null and empty blobs, sibling fields, empty results, and single-row batches). Not run against remote tables. |
||
 Avanish Garg
|
bc4c32b779 |
fix: keep tagged versions when pruning in optimize (#4544)
`optimize()` fails once a tagged version is older than the cleanup cutoff, although the docs say tagged versions are exempt from cleanup, and the failed prune also skips the index-optimize step (repro and numbers in the issue). Change: `OptimizeAction::Prune` already exposes `error_if_tagged_old_versions: Option<bool>`, so no new API. `cleanup_old_versions` and `cleanup_old_versions_before` in `rust/lancedb/src/table/optimize.rs` now use `unwrap_or(false)`, which covers `OptimizeAction::Prune`, `OptimizeAction::All` and `optimize_prune_before`. The Python and Node bindings pass `None`, which now means "retain tagged versions"; I left `python/src/table.rs` and `nodejs/src/table.rs` untouched (this also avoids a conflict with #3870, which edits that region). `Some(true)` still opts in to the error. Doc comments updated; the sentence in `docs/web/management/table.mdx` is accurate once this lands. Tests: Rust `test_optimize_prune_retains_tagged_versions` (`Some(true)` still errors; the default retains the tag; `optimize_prune_before`; `checkout_tag` plus a data read), a Python sync test with a scalar index (index step runs, a second optimize works, untagged versions are pruned, `checkout("release-1")` works), a Python async test and a Node test. **What I could and could not verify (please read):** - I could NOT compile Rust here (not enough disk), so `cargo check`, `cargo test`, the Python/Node binding builds and therefore the fixed-path assertions of the new Rust, Python and Node tests have not been run. This is why the PR is a draft; I will mark it ready once CI is green and fix whatever it finds. - Ran: both new Python tests fail on the released 0.40.0 wheel with the exact error above (so they do catch the bug); pylance 13.0.0 confirms the semantics (`cleanup_old_versions(error_if_tagged_old_versions=False)` removes the untagged versions and keeps the tagged one, still readable with `lance.dataset(version="keep")`); in Lance v14.0.0-beta.7 `cleanup.rs` the policy field is `bool`, the builder default is `true`, and Lance's own `auto_cleanup_hook` uses `Some(false)`; `rustfmt --edition 2024 --check` passes on `optimize.rs`. - The Rust and Node tests rely on timestamp ordering (`nextMillisecond()`, `days(0)`/`now()`), so they could be flaky: tell me if you prefer another approach. Behaviour change: anyone who relied on the error to learn that a tag blocks cleanup now gets a silent retain. Not changed: the deprecated sync `LanceTable.cleanup_old_versions` calls pylance directly and still errors on tagged versions (the keyword may not exist in the oldest supported pylance); `freshness.rs:1377` and `refresh.rs:1077` still pass `None` (I did not check what those paths do with tagged versions). Fixes #4543 |
||
 Aviral Saxena
|
d826b6115a |
fix(rust): explain client-side model files when remote tokenize fails (#4519)
Addresses option 2 of #4440. tokenize() builds the FTS analyzer on the client. For jieba/* and lindera/* that needs model files under LANCE_LANGUAGE_MODEL_HOME, and when they’re missing you only got lance’s raw Invalid directory path: ... error, even though server-side search works. Now, if a jieba/* or lindera/* tokenizer can’t be built, the error keeps the original cause and adds that tokenize() runs on the client, which directory the files need to be in, and that for remote tables they must match the server’s. I also updated the tokenize docs (Rust, Python, TS, generated JS docs) to say a mismatch can’t be detected, so tokens may differ from the server’s index. Behavior when the files exist is unchanged. Detecting a mismatch needs a server-side tokenize endpoint (option 1), so I left it out. The hint isn’t limited to remote tables, since Table::as_native is slated for removal and the wording fits local tables too. Tests: Rust test with mocked jieba/default index_details and an empty model dir, one checking other tokenizer errors don’t get the hint, and Python sync/async tests. |
||
  adenzhou1350andaden
|
b205e7c628 |
fix(node): prevent table listing limits from wrapping (#4492)
## Summary Prevent large JavaScript table-listing limits from wrapping during the native `u32` conversion. With two tables, `limit: 2 ** 32` currently returns no tables and `2 ** 32 + 1` returns only one. Validate nonnegative safe integers using the existing helper and clamp valid limits above `u32::MAX` before calling the native binding. Apply the same normalization to `tableNames` and `listTables`, including namespace overloads. Undefined, zero, small limits and `u32::MAX` keep their existing behavior. ## Validation Built the real Node native extension from main `0aa842b17f13d1f8ffd2851bc57c222bc5a376ba` with the repository's `pnpm build`, using locked dependencies and reusable build caches. No mocked/released native extension or disabled TypeScript diagnostics. With identical regression tests and that native library in fresh Jest processes: - Baseline: 40 passed, 8 failed; candidate: **48 passed**, no failures. - Both arms: `pnpm run tsc` and `pnpm run docs` passed. - Changed-file Biome check passed. Tests cover wrapping limits, invalid numbers, zero/small/u32 limits, defaults and both overload forms. Validation is the focused connection suite, not the complete repository suite. No performance claim. AI-assisted implementation and verification; the patch and exact tested source bytes were reviewed before publication. Signed-off-by: aden <aden@example.com> Co-authored-by: aden <aden@example.com> |
||
|
adenzhou1350
|
13d54db594 |
fix(node): preserve caller-owned storage options (#4480)
`getStorageOptions` writes typed creation options back into the caller's
`storageOptions` object. This makes frozen options throw and causes
overrides to leak into later calls that reuse the storage map.
For example, using `{ storageOptions: shared, enableV2ManifestPaths:
true }` changes `shared.newTableEnableV2ManifestPaths` to `"true"`. A
subsequent creation with just `{ storageOptions: shared }` then uses V2
manifest paths even if `shared` originally requested `"false"`.
Merge the typed overrides into a local map instead. Their existing
precedence, key normalization, and undefined/default behavior remain
unchanged. This affects both `createTable` and `createEmptyTable`; no
native/Rust changes.
Adds six native-database regressions covering frozen creation options,
frozen nested storage options with conflicting defaults, and cross-call
reuse for both APIs.
Validation (Linux x64 / Node 24.19.0 / official 0.41.0-beta.0 native
library):
- Before: all six new regressions fail; an existing namespace
storage-options control passes.
- After: the full connection test file passes, 43/43 (current TypeScript
source, transpile-only).
- Same fix on the matching 0.41.0-beta.0 source: TypeScript compilation,
strict ts-jest connection tests (43/43), and TypeDoc with
warnings-as-errors pass.
- Biome on changed files and whitespace checks pass.
A current-main native rebuild was not run. Released native declarations
predate six unrelated lazy-iterator signatures; current-main
typechecking reports the same six errors before and after this fix (only
downstream line coordinates shift).
### Connection entry-point follow-up
The same ownership problem occurs in `connect(uri, options)` and
`connectNamespace`: normalization assigns a new storage map into the
caller's options. Copy the options before normalization instead. The
`connect(options)` overload already copies its input and is retained as
a control.
Nine added native-database cases cover frozen outer options
(with/without storage options) and repeated reuse, including table
creation and row-count checks. Before this entry-point fix, the
URI/namespace cases fail (6), while the object-overload controls pass
(3). Afterward the complete connection file passes 52/52 on current
TypeScript with the official 0.41.0-beta.0 native library. Applying the
entry-point fix to matching-release source also passes strict
TypeScript, strict ts-jest (52/52), and TypeDoc. Biome passes on both
newly changed files.
The same six current-main/released-native lazy-iterator declaration
errors remain unchanged. No Rust/native rebuild or GPU claim is added.
---------
Signed-off-by: Xucheng Zhou <aden1350@outlook.com>
|