Bump version: 0.34.0-beta.7 → 0.34.0

.agents/plugins/marketplace.json

@@ -1,20 +0,0 @@
-{
-  "name": "lancedb",
-  "interface": {
-    "displayName": "LanceDB"
-  },
-  "plugins": [
-    {
-      "name": "lancedb",
-      "source": {
-        "source": "local",
-        "path": "./plugins/lancedb"
-      },
-      "policy": {
-        "installation": "AVAILABLE",
-        "authentication": "ON_INSTALL"
-      },
-      "category": "Developer Tools"
-    }
-  ]
-}

.agents/skills/README.md

@@ -5,7 +5,3 @@ This directory contains repo-scoped code agent skills for the LanceDB project.
 Each skill is a folder that contains a required `SKILL.md` and optional bundled resources.
 
 Codex discovers skills from `.agents/skills` in the current working directory and parent directories.
-
-The `lancedb` skill lives in the `plugins/lancedb` plugin (see `plugins/lancedb/skills/lancedb`)
-so it can be installed via the plugin marketplaces (`.claude-plugin/marketplace.json` and
-`.agents/plugins/marketplace.json`); the `lancedb` entry here is a symlink into that plugin.

.agents/skills/lancedb

@@ -1 +0,0 @@
-../../plugins/lancedb/skills/lancedb

.agents/skills/lancedb-branch-ops/SKILL.md

@@ -0,0 +1,137 @@
+---
+name: lancedb-branch-ops
+description: Branch management for LanceDB tables via the REST API. Use this skill whenever someone wants to create, delete, list, or switch branches on a LanceDB table — or needs to make sure a write (metadata update, index build, etc.) lands on a specific branch instead of main. Invoke it even without the word "branch" if context makes clear they want an experimental copy of a table, want to isolate changes, or want to confirm a mutation didn't touch main. Covers: branches/list, branches/create, branches/delete, and passing "branch" in describe/update_field_metadata/create_index to target a non-main version.
+---
+
+## Goal
+
+Manage branches on a LanceDB table: list what exists, create new ones, delete stale ones, and direct read/write operations at a specific branch without touching main.
+
+## Step 0: Establish the connection
+
+Use the `lancedb-connect` skill to resolve the base URL and auth headers (`x-api-key`, `x-lancedb-database`). Skip this only if the connection is already known from the current conversation.
+
+All examples below use `{base_url}` — substitute the resolved endpoint and include the auth headers on every request.
+
+## The branch model (important)
+
+LanceDB branches are named snapshots that diverge from the table's current state at creation time. There is **no checkout command** — you never switch the whole table to a branch. Instead, you **pass `"branch": "<name>"` in the request body** of any operation to target that branch. Omitting the key (or sending an empty body) always targets main.
+
+`branches/list` returns only non-main branches. Main always exists and is not listed.
+
+## List branches
+
+```http
+POST {base_url}/v1/table/{table_id}/branches/list
+Content-Type: application/json
+
+{}
+```
+
+Response:
+```json
+{
+  "branches": {
+    "experiment-reindex": {"parentVersion": 1, "createAt": 1782506085, "manifestSize": 1029}
+  }
+}
+```
+
+If `branches` is `{}`, the table has no branches besides main.
+
+## Create a branch
+
+```http
+POST {base_url}/v1/table/{table_id}/branches/create
+Content-Type: application/json
+
+{"name": "experiment-reindex"}
+```
+
+HTTP 200 with `{}` body = success. The branch is created off the table's current state on main.
+
+Verify by calling `branches/list` and confirming the new name appears.
+
+## Delete a branch
+
+```http
+POST {base_url}/v1/table/{table_id}/branches/delete
+Content-Type: application/json
+
+{"name": "stale-2024"}
+```
+
+HTTP 200 with `{}` body = success. Only the branch pointer is removed — main and all row data remain intact.
+
+Verify by calling `branches/list` (name gone) and `describe` with no branch param (main still responds).
+
+## Operate on a specific branch
+
+Pass `"branch": "<name>"` in the body of any operation to scope it to that branch:
+
+**Read schema on a branch:**
+```http
+POST {base_url}/v1/table/{table_id}/describe
+Content-Type: application/json
+
+{"branch": "wip-branch"}
+```
+
+**Write metadata to a branch (not main):**
+```http
+POST {base_url}/v1/table/{table_id}/update_field_metadata
+Content-Type: application/json
+
+{
+  "branch": "wip-branch",
+  "updates": [
+    {
+      "path": "category",
+      "metadata": {"lancedb:description": "Product category label."},
+      "replace": false
+    }
+  ]
+}
+```
+
+**Build an index on a branch:**
+```http
+POST {base_url}/v1/table/{table_id}/create_index
+Content-Type: application/json
+
+{
+  "branch": "wip-branch",
+  "column": "category",
+  "index_type": "BTREE"
+}
+```
+
+## Verifying isolation
+
+After writing to a branch, always confirm the change did NOT land on main:
+
+```bash
+# Should show the new metadata
+curl -s -X POST {base_url}/v1/table/{table_id}/describe \
+  -H "x-api-key: <key>" -H "x-lancedb-database: <db>" \
+  -H "content-type: application/json" \
+  -d '{"branch": "wip-branch"}'
+
+# Should NOT show the new metadata
+curl -s -X POST {base_url}/v1/table/{table_id}/describe \
+  -H "x-api-key: <key>" -H "x-lancedb-database: <db>" \
+  -H "content-type: application/json" \
+  -d '{}'
+```
+
+## Quick reference
+
+| Goal | Endpoint | Body |
+|------|----------|------|
+| List all branches | `branches/list` | `{}` |
+| Create a branch | `branches/create` | `{"name": "..."}` |
+| Delete a branch | `branches/delete` | `{"name": "..."}` |
+| Read schema on branch | `describe` | `{"branch": "..."}` |
+| Write metadata on branch | `update_field_metadata` | `{"branch": "...", "updates": [...]}` |
+| Build index on branch | `create_index` | `{"branch": "...", "column": ..., "index_type": ...}` |
+| Target main (default) | any endpoint | omit `"branch"` key |

.agents/skills/lancedb-column-metadata/SKILL.md

@@ -0,0 +1,178 @@
+---
+name: lancedb-column-metadata
+description: Column metadata authoring for LanceDB tables via the REST API. This skill is required for tasks like writing field descriptions, setting tags on columns (field_type, model, project_id, version), classifying columns as embeddings vs labels vs eval metrics, or grouping versioned columns into logical families — because it has the API integration needed to read the schema and persist metadata back. Invoke whenever someone wants to document, annotate, tag, or classify what their table columns ARE. Trigger even without an explicit "LanceDB" mention, as long as the context is column-level documentation or tagging for an ML or vector database table.
+metadata:
+  short-description: Write column descriptions, tags, and logical groupings to a LanceDB table
+---
+
+## Overview
+
+This skill authors column-level metadata for a LanceDB table. It connects to a LanceDB deployment over its REST API, inspects the table schema, generates appropriate metadata, and writes it back.
+
+## Step 0: Establish the connection
+
+Use the `lancedb-connect` skill (invoke it via the Skill tool) to resolve the base URL and auth headers (`x-api-key`, `x-lancedb-database`) for whichever deployment the user is working against — enterprise/self-hosted or a local dev server. Skip it only if the connection details are already established in the conversation.
+
+All examples below use `{base_url}` — substitute the resolved endpoint and include the resolved headers on every request.
+
+## Metadata keys
+
+All metadata uses namespaced keys:
+
+| Key | Purpose | Example value |
+|-----|---------|---------------|
+| `lancedb:description` | Human-readable explanation of what the column contains | `"CLIP ViT-L/14 image embedding, L2-normalized (768-dim)"` |
+| `lancedb:tag:<name>` | Flexible key-value tag; the suffix names the tag category | `lancedb:tag:field_type: "embedding"`, `lancedb:tag:model: "clip"`, `lancedb:tag:project_id: "foo"` |
+| `lancedb:logical-column` | Logical group/family this column belongs to | `"clip_features"` |
+
+Tags are open-ended — use whatever key suffix and value make sense given the user's intent. The tag suffix should describe *what is being classified* (e.g., `field_type`, `model`, `project_id`) and the value describes *how*.
+
+## Step 1: Resolve the table identifier
+
+You need:
+- **Table name** (required) — e.g., `my_table` or `my_namespace.my_table`
+- **Database name** — ask if not provided and not inferable from context; it goes in the `x-lancedb-database` header, never in the URL path
+
+The table identifier in the URL path is typically `table_name` for a top-level table, or `namespace$table_name` if the table lives in a namespace. The API accepts a `delimiter` query parameter to parse compound identifiers (default `$`).
+
+## Step 2: Describe the table
+
+```http
+POST {base_url}/v1/table/{table_id}/describe
+Content-Type: application/json
+
+{}
+```
+
+The response contains `schema.fields` — an array of field objects:
+
+```json
+{
+  "schema": {
+    "fields": [
+      {
+        "name": "clip_embedding_v3",
+        "type": { "type": "FixedSizeList", "fields": [...], "listSize": 768 },
+        "nullable": true,
+        "metadata": { "lancedb:description": "..." }
+      }
+    ]
+  }
+}
+```
+
+Each field has:
+- `name` — field name
+- `type` — Arrow data type (check `type.type` for the type string)
+- `nullable` — boolean
+- `metadata` — existing key-value metadata (read this before writing to avoid redundant updates)
+
+For struct/nested fields, recurse into `type.fields` and represent them as dot-notation paths (e.g., `parent.child`).
+
+If the user hasn't specified which columns to update, work with all columns.
+
+## Step 3: Generate metadata
+
+Decide what to generate based on the user's request.
+
+### Writing descriptions (`lancedb:description`)
+
+Base descriptions on:
+- The column name and Arrow type (e.g., `FixedSizeList` of floats → likely an embedding)
+- User-supplied context (upstream pipeline, sample values, domain knowledge)
+- Name patterns: `_embedding`/`_vec`/`_embed` → vector; `_label`/`_class` → label; `_score`/`_eval`/`_metric` → evaluation metric
+
+Be specific and concise. Good: `"Sentence-BERT embedding of the query text (768-dim)."` Not: `"An embedding column."`
+
+### Tagging columns (`lancedb:tag:<name>`)
+
+Choose tag key names that match what the user asked to annotate. Common patterns:
+
+- Semantic field type → `lancedb:tag:field_type: "embedding"` / `"text"` / `"image"` / `"label"` / `"eval"` / `"id"` / `"metadata"`
+- Model or source → `lancedb:tag:model: "clip"` / `"bert"` / `"vit"`
+- Project affiliation → `lancedb:tag:project_id: "<name>"`
+- Version → `lancedb:tag:version: "v3"` (and `lancedb:tag:latest: "true"` for the newest)
+
+Use Arrow type as a hint: `FixedSizeList` + float → embedding; `Utf8`/`LargeUtf8` → text; `Binary` → image or blob.
+
+Multiple tags on the same column are fine — each is a separate key.
+
+### Grouping into logical columns (`lancedb:logical-column`)
+
+Look for naming patterns across columns:
+- `clip_v1`, `clip_v2`, `clip_v3` → logical column `"clip"`, latest is `v3`
+- `text_embed_20240101`, `text_embed_20240601` → logical column `"text_embed"`, latest is the most recent date suffix
+
+Write `lancedb:logical-column` on all members of a group. Mark the newest with `lancedb:tag:latest: "true"` (in addition to its version tag).
+
+## Step 4: Write the metadata
+
+```http
+POST {base_url}/v1/table/{table_id}/update_field_metadata
+Content-Type: application/json
+
+{
+  "updates": [
+    {
+      "path": "clip_v3",
+      "metadata": {
+        "lancedb:description": "CLIP ViT-L/14 image embedding, L2-normalized (1024-dim).",
+        "lancedb:tag:field_type": "embedding",
+        "lancedb:tag:model": "clip",
+        "lancedb:tag:version": "v3",
+        "lancedb:tag:latest": "true",
+        "lancedb:logical-column": "clip"
+      },
+      "replace": false
+    },
+    {
+      "path": "clip_v2",
+      "metadata": {
+        "lancedb:description": "CLIP ViT-B/32 image embedding (768-dim), superseded by v3.",
+        "lancedb:tag:field_type": "embedding",
+        "lancedb:tag:model": "clip",
+        "lancedb:tag:version": "v2",
+        "lancedb:logical-column": "clip"
+      },
+      "replace": false
+    }
+  ]
+}
+```
+
+Rules:
+- **Use `"replace": false`** (merge) by default — this preserves existing metadata the user didn't ask to change
+- Use `"replace": true` only if the user explicitly asks to overwrite all existing metadata on a column
+- Set a value to `null` to delete a specific key
+- Batch all updates in a single request when possible
+
+The response includes `version` (new table version) and `fields` (the updated metadata per field).
+
+## Step 5: Confirm
+
+Report back:
+- Which columns were updated and what was written
+- The new table version number
+- Any columns skipped (e.g., already had up-to-date metadata)
+
+---
+
+## Quick examples
+
+**"Write descriptions for all columns in the `product_embeddings` table"**
+1. POST `/v1/table/product_embeddings/describe` → get all fields
+2. Generate a `lancedb:description` for each column based on name + type
+3. POST `update_field_metadata` with descriptions
+4. Report
+
+**"Tag the columns in `model_outputs` with their field type and model"**
+1. Describe `model_outputs`
+2. For each field, classify by name + Arrow type → set `lancedb:tag:field_type` and `lancedb:tag:model` where applicable
+3. POST `update_field_metadata`
+4. Report
+
+**"Group the feature columns in `training_features` into logical families and mark the latest version"**
+1. Describe the table
+2. Find version patterns → assign `lancedb:logical-column` and `lancedb:tag:version`; mark newest with `lancedb:tag:latest: "true"`
+3. POST `update_field_metadata`
+4. Show the grouping

.agents/skills/lancedb-connect/SKILL.md

@@ -0,0 +1,42 @@
+---
+name: lancedb-connect
+description: Resolve how to connect to a LanceDB deployment over the REST API — figure out the base URL, API key, and database header. Use this before making any REST requests to a LanceDB table, whenever the endpoint or auth setup is not already known. Also useful on its own when someone asks how to connect, authenticate, or curl their LanceDB instance.
+metadata:
+  short-description: Resolve the base URL and auth headers for a LanceDB deployment
+---
+
+## Goal
+
+Produce two things every REST request needs:
+
+1. **Base URL** — the endpoint
+2. **Headers** — `x-api-key`, and usually `x-lancedb-database`
+
+## Resolution steps
+
+1. If the user already gave a URL and API key (or said which environment they're working against), use that.
+2. Otherwise, look for credentials already available in the environment:
+   - Env vars like `LANCEDB_URI` / `LANCEDB_HOST` / `LANCEDB_API_KEY`
+   - A LanceDB endpoint already running or port-forwarded locally (the REST default port is 2333, i.e. `http://localhost:2333`)
+3. If you didn't find both pieces, ask the user directly: **"What's your LanceDB endpoint's URL, and what's your API key?"** Also ask which database to use if it isn't obvious. Don't guess or probe further — the user knows their deployment.
+
+## Validating the connection
+
+Make a cheap authenticated request and check the status:
+
+```bash
+curl -s -w "\n%{http_code}" "{base_url}/v1/table/?limit=1" \
+  -H "x-api-key: <key>" \
+  -H "x-lancedb-database: <database>"
+```
+
+- `200` — connection, key, and database header all good
+- `401` — API key missing or wrong
+- `400` mentioning a database header — this deployment expects `x-lancedb-database`
+
+## Non-REST equivalents
+
+If the caller would rather use the SDK or CLI than raw REST, the same credentials work:
+
+- Python SDK: `lancedb.connect("db://<database>", api_key="<key>", host_override="<base_url>")`
+- `lancedb` CLI: a `[profiles.<name>]` entry in `~/.lancedb/config.toml` with `http_server_url`, `api_key`, `database`

.bumpversion.toml

@@ -1,5 +1,5 @@
 [tool.bumpversion]
-current_version = "0.37.1"
+current_version = "0.31.0-beta.6"
 parse = """(?x)
     (?P<major>0|[1-9]\\d*)\\.
     (?P<minor>0|[1-9]\\d*)\\.
@@ -75,13 +75,6 @@ filename = "nodejs/Cargo.toml"
 replace = "\nversion = \"{new_version}\""
 search = "\nversion = \"{current_version}\""
 
-# The Python package takes its version from here (pyproject.toml declares
-# `dynamic = ["version"]`, so maturin reads it out of the crate manifest).
-[[tool.bumpversion.files]]
-filename = "python/Cargo.toml"
-replace = "\nversion = \"{new_version}\""
-search = "\nversion = \"{current_version}\""
-
 # Java documentation
 [[tool.bumpversion.files]]
 filename = "docs/src/java/java.md"

.claude-plugin/marketplace.json

@@ -1,19 +0,0 @@
-{
-  "name": "lancedb",
-  "owner": {
-    "name": "LanceDB"
-  },
-  "description": "LanceDB plugins for Claude Code.",
-  "plugins": [
-    {
-      "name": "lancedb",
-      "source": "./plugins/lancedb",
-      "description": "Write, review, debug, and document LanceDB pipelines in Python and TypeScript that work across local LanceDB OSS tables and remote LanceDB Enterprise/Cloud tables.",
-      "version": "0.1.0",
-      "author": {
-        "name": "LanceDB"
-      },
-      "category": "development"
-    }
-  ]
-}

.github/actions/create-failure-issue/action.yml

@@ -27,31 +27,19 @@ runs:
           # Extract failed job names
           FAILED_JOBS=$(echo "$JOB_RESULTS" | jq -r 'to_entries | map(select(.value.result == "failure")) | map(.key) | join(", ")')
 
-          TITLE="$WORKFLOW_NAME Failed ($FAILED_JOBS)"
-
-          # This action now also runs on nightly schedules, so a breakage that
-          # persists for a few days would otherwise file one issue per night.
-          # Comment on the open report instead when one already exists.
-          EXISTING=$(gh issue list --state open --label ci --limit 100 --json number,title \
-            | jq -r --arg title "$TITLE" 'map(select(.title == $title)) | .[0].number // empty')
-
-          if [ -n "$EXISTING" ]; then
-            gh issue comment "$EXISTING" --body "Failed again: $RUN_URL"
-            echo "Commented on existing issue #$EXISTING"
-          else
-            gh issue create \
-              --title "$TITLE" \
-              --body "The workflow **$WORKFLOW_NAME** failed during execution.
+          # Create issue with workflow name, failed jobs, and run URL
+          gh issue create \
+            --title "$WORKFLOW_NAME Failed ($FAILED_JOBS)" \
+            --body "The workflow **$WORKFLOW_NAME** failed during execution.
 
         **Failed jobs:** $FAILED_JOBS
 
         **Run URL:** $RUN_URL
 
         Please investigate the failed jobs and address any issues." \
-              --label "ci"
+            --label "ci"
 
-            echo "Issue created successfully"
-          fi
+          echo "Issue created successfully"
         else
           echo "No job failures detected, skipping issue creation"
         fi

.github/workflows/build_linux_wheel/action.yml

@@ -18,14 +18,6 @@ inputs:
     description: "The manylinux version to build for"
     required: false
     default: "2_17"
-  package-name:
-    description: "Override [project] name in python/pyproject.toml (e.g. 'lancedb-compat'). Default keeps 'lancedb'."
-    required: false
-    default: "lancedb"
-  rustflags:
-    description: "RUSTFLAGS for the build container, as a single whitespace-free token (e.g. '-Ctarget-cpu=x86-64-v2'). Empty leaves RUSTFLAGS unset, keeping the defaults from .cargo/config.toml."
-    required: false
-    default: ""
 runs:
   using: "composite"
   steps:
@@ -35,18 +27,6 @@ runs:
         ARM_BUILD: ${{ inputs.arm-build }}
       run: |
         echo "ARM BUILD: $ARM_BUILD"
-    - name: Patch package name for variant build
-      if: ${{ inputs.package-name != 'lancedb' }}
-      shell: bash
-      env:
-        PACKAGE_NAME: ${{ inputs.package-name }}
-      run: |
-        # Swap the [project] name so this build produces e.g. lancedb-compat
-        # wheels. The package still installs files under the lancedb/
-        # namespace -- import lancedb still works after pip install.
-        sed -i.bak 's/^name = "lancedb"$/name = "'"$PACKAGE_NAME"'"/' python/pyproject.toml
-        rm -f python/pyproject.toml.bak
-        grep '^name = ' python/pyproject.toml
     - name: Build x86_64 Manylinux wheel
       if: ${{ inputs.arm-build == 'false' }}
       uses: PyO3/maturin-action@v1
@@ -54,16 +34,15 @@ runs:
         maturin-version: "1.12.4"
         command: build
         working-directory: python
-        docker-options: "-e PIP_EXTRA_INDEX_URL='https://pypi.fury.io/lance-format/ https://pypi.fury.io/lancedb/' -e PROTOC=/usr/local/bin/protoc ${{ inputs.rustflags != '' && format('-e RUSTFLAGS={0}', inputs.rustflags) || '' }}"
+        docker-options: "-e PIP_EXTRA_INDEX_URL='https://pypi.fury.io/lance-format/ https://pypi.fury.io/lancedb/'"
         target: x86_64-unknown-linux-gnu
         manylinux: ${{ inputs.manylinux }}
         args: ${{ inputs.args }}
         before-script-linux: |
           set -e
-          curl -fsSL https://github.com/protocolbuffers/protobuf/releases/download/v24.4/protoc-24.4-linux-x86_64.zip -o /tmp/protoc.zip
-          unzip /tmp/protoc.zip -d /usr/local
-          rm /tmp/protoc.zip
-          /usr/local/bin/protoc --version
+          curl -L https://github.com/protocolbuffers/protobuf/releases/download/v24.4/protoc-24.4-linux-$(uname -m).zip > /tmp/protoc.zip \
+            && unzip /tmp/protoc.zip -d /usr/local \
+            && rm /tmp/protoc.zip
     - name: Build Arm Manylinux Wheel
       if: ${{ inputs.arm-build == 'true' }}
       uses: PyO3/maturin-action@v1
@@ -71,14 +50,13 @@ runs:
         maturin-version: "1.12.4"
         command: build
         working-directory: python
-        docker-options: "-e PIP_EXTRA_INDEX_URL='https://pypi.fury.io/lance-format/ https://pypi.fury.io/lancedb/' -e PROTOC=/usr/local/bin/protoc ${{ inputs.rustflags != '' && format('-e RUSTFLAGS={0}', inputs.rustflags) || '' }}"
+        docker-options: "-e PIP_EXTRA_INDEX_URL='https://pypi.fury.io/lance-format/ https://pypi.fury.io/lancedb/'"
         target: aarch64-unknown-linux-gnu
         manylinux: ${{ inputs.manylinux }}
         args: ${{ inputs.args }}
         before-script-linux: |
           set -e
-          yum install -y clang
-          curl -fsSL https://github.com/protocolbuffers/protobuf/releases/download/v24.4/protoc-24.4-linux-aarch_64.zip -o /tmp/protoc.zip
-          unzip /tmp/protoc.zip -d /usr/local
-          rm /tmp/protoc.zip
-          /usr/local/bin/protoc --version
+          yum install -y clang \
+            && curl -L https://github.com/protocolbuffers/protobuf/releases/download/v24.4/protoc-24.4-linux-aarch_64.zip > /tmp/protoc.zip \
+            && unzip /tmp/protoc.zip -d /usr/local \
+            && rm /tmp/protoc.zip

.github/workflows/cargo-publish.yml

@@ -6,6 +6,7 @@ on:
       # We don't publish pre-releases for Rust. Crates.io is just a source
       # distribution, so we don't need to publish pre-releases.
       - "v*-beta*"
+      - "*-v*" # for example, python-vX.Y.Z
 
 env:
   # This env var is used by Swatinem/rust-cache@v2 for the cache

.github/workflows/docs-link-check.yml

@@ -1,222 +0,0 @@
-name: Check doc links
-
-# Checking external links is inherently noisy: third-party sites rate-limit
-# automated clients, reject non-browser user agents, and go down temporarily.
-# Blocking pull requests on that trades a lot of false failures for very little
-# signal, so this runs on a schedule and reports findings in a single tracking
-# issue instead of failing anyone's build.
-on:
-  schedule:
-    - cron: "0 7 * * *"
-  workflow_dispatch:
-
-# The report lives in one repository-global issue, so runs must not overlap: a
-# lookup racing a create produces duplicate issues, and a healthy run closing
-# the issue while a failing run only rewrites its body would leave a broken
-# report closed. The group is deliberately ref-independent so that a manual
-# dispatch serializes against the scheduled run.
-concurrency:
-  group: docs-link-check
-  cancel-in-progress: false
-
-permissions: {}
-
-env:
-  REPORT_TITLE: "Docs link checker report"
-
-jobs:
-  scan:
-    name: Scan links
-    runs-on: ubuntu-24.04
-    # lychee-action is pinned by SHA, but its wrapper downloads the lychee
-    # release tarball at run time without verifying a digest, and hands the
-    # resulting binary a GitHub token. Release assets remain replaceable, so
-    # that binary is confined to a job whose token can only read public
-    # content; everything that writes runs in the report job below.
-    permissions:
-      contents: read
-    outputs:
-      exit_code: ${{ steps.lychee.outputs.exit_code }}
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v6
-        with:
-          # workflow_dispatch can run from any ref, but the report is
-          # repository-global. Always measure the default branch so a manual
-          # run from a topic branch cannot close a report that main warrants,
-          # or overwrite it with branch-only findings.
-          ref: ${{ github.event.repository.default_branch }}
-          persist-credentials: false
-
-      - name: Check links
-        id: lychee
-        uses: lycheeverse/lychee-action@e7477775783ea5526144ba13e8db5eec57747ce8 # v2.9.0
-        with:
-          # Restricted to http(s) on purpose. Much of docs/src is generated
-          # API reference (the js/ tree comes from `npm run docs` in nodejs)
-          # and the hand-written pages use mkdocstrings cross-references and
-          # nav-relative paths that only resolve in the site mkdocs builds,
-          # not in this checkout, so relative links would be reported as
-          # broken on every run.
-          args: >-
-            --scheme https
-            --scheme http
-            --no-progress
-            --max-retries 3
-            --timeout 20
-            'docs/src/**/*.md'
-          format: json
-          output: ./lychee/out.json
-          jobSummary: false
-          # The report, not a red build, is the signal for broken links. The
-          # validation step below still fails the run if the check itself
-          # breaks.
-          fail: false
-
-      - name: Validate report
-        # lychee does not reserve exit code 2 for broken links: its CLI
-        # parser also exits 2 on an invalid option, before any link was
-        # checked or any report written. Only a parseable report whose
-        # counts agree with the exit code counts as a link verdict; anything
-        # else fails here, and the report job below is skipped entirely, so
-        # the tracking issue is never touched. Exit 2 covers timeouts as
-        # well as errors, and a timed-out host is exactly the transient
-        # unavailability this report exists to surface, so both count as
-        # findings. Requiring total > 0 also catches a glob that silently
-        # stopped matching any file.
-        if: steps.lychee.outputs.exit_code == 0 || steps.lychee.outputs.exit_code == 2
-        env:
-          EXIT_CODE: ${{ steps.lychee.outputs.exit_code }}
-        run: |
-          jq -e --argjson code "$EXIT_CODE" '
-            (.total > 0) and
-            (if $code == 0
-             then .errors == 0 and .timeouts == 0
-               and (.error_map | length == 0) and (.timeout_map | length == 0)
-             else (.errors + .timeouts) > 0
-               and ((.error_map | length) + (.timeout_map | length)) > 0
-             end)
-          ' ./lychee/out.json
-
-      - name: Upload report
-        if: steps.lychee.outputs.exit_code == 2
-        uses: actions/upload-artifact@v7
-        with:
-          name: link-report
-          path: ./lychee/out.json
-          retention-days: 7
-
-  report:
-    name: Update report issue
-    needs: scan
-    runs-on: ubuntu-24.04
-    # Deliberately no checkout: this job needs the report artifact and the
-    # issues API, not the repository contents.
-    permissions:
-      issues: write
-    env:
-      EXIT_CODE: ${{ needs.scan.outputs.exit_code }}
-      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-    steps:
-      - name: Classify checker result
-        # lychee exits 0 when every link resolves and 2 when links fail,
-        # both already cross-checked against the report by the scan job's
-        # validation step. Anything else (1 runtime, 3 bad config) means the
-        # check never produced a link verdict, which must surface as a failed
-        # run rather than be published as "broken documentation links".
-        run: |
-          case "$EXIT_CODE" in
-            0|2)
-              echo "lychee exit code $EXIT_CODE"
-              ;;
-            *)
-              echo "::error::lychee exited with '$EXIT_CODE': the link check did not complete. Leaving the report issue untouched."
-              exit 1
-              ;;
-          esac
-
-      - name: Find existing report issue
-        id: report
-        # Matched on title alone, and through search rather than a listing:
-        # the issue action applies labels in a separate call after creating the
-        # issue, so a label filter misses a half-created report, and this
-        # repository has far more open issues than one listing page holds.
-        # Closed issues are included because a healthy run closes the report:
-        # an open-only lookup would forget that identity and the next failing
-        # run would open a duplicate. The oldest match stays the canonical
-        # report and is reopened below when links break again.
-        run: |
-          match=$(gh issue list --repo "$GITHUB_REPOSITORY" --state all \
-            --search "in:title \"$REPORT_TITLE\" author:app/github-actions" \
-            --limit 50 --json number,title,state \
-            --jq "[.[] | select(.title == \"$REPORT_TITLE\")] | sort_by(.number) | first // empty")
-          echo "number=$(jq -r '.number // empty' <<<"$match")" >> "$GITHUB_OUTPUT"
-          echo "state=$(jq -r '.state // empty' <<<"$match")" >> "$GITHUB_OUTPUT"
-
-      - name: Download report
-        if: env.EXIT_CODE == 2
-        uses: actions/download-artifact@v8
-        with:
-          name: link-report
-          path: ./lychee
-
-      - name: Compose report
-        if: env.EXIT_CODE == 2
-        run: |
-          run_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
-          {
-            echo "Broken documentation links found by [\`$GITHUB_WORKFLOW\`]($run_url)."
-            echo
-            echo "This issue is rewritten by every scheduled run and closed automatically once all links resolve."
-            echo
-            echo "Entries can be false positives: some sites rate-limit or block automated clients while working fine in a browser. Confirm before editing the docs, and add persistent offenders to \`--exclude\` in \`.github/workflows/docs-link-check.yml\`."
-            echo
-            # Timeouts are reported alongside errors: entries land in
-            # timeout_map with a status text instead of an HTTP code.
-            jq -r '
-              "\(.errors) of \(.total) links failed, \(.timeouts) timed out.",
-              "",
-              ([(.error_map | to_entries[]), (.timeout_map | to_entries[])]
-               | group_by(.key)[] |
-                "### Errors in \(.[0].key)",
-                "",
-                (map(.value[])[] | "* [\(.status.code // .status.text // "ERR")] <\(.url)> — \(.status.details // .status.text // "unknown error")"),
-                "")
-            ' ./lychee/out.json
-          } > ./lychee/issue.md
-
-      - name: Reopen report issue
-        # A healthy run closes the report, and the issue action below only
-        # rewrites the body of whatever number it is given. Without an
-        # explicit reopen, the 2 -> 0 -> 2 sequence would keep rewriting a
-        # closed issue while links are broken. A CLOSED state implies the
-        # lookup found a canonical issue, so no separate emptiness check.
-        if: env.EXIT_CODE == 2 && steps.report.outputs.state == 'CLOSED'
-        env:
-          ISSUE_NUMBER: ${{ steps.report.outputs.number }}
-        run: |
-          run_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
-          gh issue reopen "$ISSUE_NUMBER" --repo "$GITHUB_REPOSITORY" \
-            --comment "Broken documentation links found again in [the latest run]($run_url)."
-
-      - name: Report broken links
-        if: env.EXIT_CODE == 2
-        uses: peter-evans/create-issue-from-file@fca9117c27cdc29c6c4db3b86c48e4115a786710 # v6.0.0
-        with:
-          # Empty on the first failing run, which creates the issue; afterwards
-          # the same issue is updated in place.
-          issue-number: ${{ steps.report.outputs.number }}
-          title: ${{ env.REPORT_TITLE }}
-          content-filepath: ./lychee/issue.md
-          labels: documentation
-
-      - name: Close report issue once links are healthy
-        # An OPEN state implies the lookup found a canonical issue; a report
-        # that is already closed needs nothing.
-        if: env.EXIT_CODE == 0 && steps.report.outputs.state == 'OPEN'
-        env:
-          ISSUE_NUMBER: ${{ steps.report.outputs.number }}
-        run: |
-          run_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
-          gh issue close "$ISSUE_NUMBER" --repo "$GITHUB_REPOSITORY" \
-            --comment "All documentation links resolved in [the latest run]($run_url)."

.github/workflows/gh-release.yml

@@ -1,85 +0,0 @@
-name: GitHub Release
-
-# All SDKs share one version, so a single `vX.Y.Z` tag produces a single GitHub
-# release covering all of them. The per-package publish workflows (PyPI, NPM,
-# Cargo, Maven) trigger off the same tag independently.
-
-on:
-  push:
-    tags:
-      - "v*"
-
-permissions:
-  contents: read
-
-jobs:
-  gh-release:
-    runs-on: ubuntu-latest
-    permissions:
-      contents: write
-    steps:
-      - uses: actions/checkout@v6
-        with:
-          fetch-depth: 0
-          lfs: true
-      - name: Extract version
-        id: extract_version
-        env:
-          GITHUB_REF: ${{ github.ref }}
-        run: |
-          set -e
-          echo "Extracting tag and version from $GITHUB_REF"
-          if [[ $GITHUB_REF =~ refs/tags/v(.*) ]]; then
-            VERSION=${BASH_REMATCH[1]}
-            TAG=v$VERSION
-            echo "tag=$TAG" >> $GITHUB_OUTPUT
-            echo "version=$VERSION" >> $GITHUB_OUTPUT
-          else
-            echo "Failed to extract version from $GITHUB_REF"
-            exit 1
-          fi
-          echo "Extracted version $VERSION from $GITHUB_REF"
-          if [[ $VERSION =~ beta ]]; then
-            echo "This is a beta release"
-            echo "prerelease=true" >> $GITHUB_OUTPUT
-
-            # Get last release (that is not this one)
-            FROM_TAG=$(git tag --sort='version:refname' \
-              | grep ^v \
-              | grep -vF "$TAG" \
-              | python ci/semver_sort.py v \
-              | tail -n 1)
-          else
-            echo "This is a stable release"
-            echo "prerelease=false" >> $GITHUB_OUTPUT
-            # Get last stable tag (ignore betas)
-            FROM_TAG=$(git tag --sort='version:refname' \
-              | grep ^v \
-              | grep -vF "$TAG" \
-              | grep -v beta \
-              | python ci/semver_sort.py v \
-              | tail -n 1)
-          fi
-          echo "Found from tag $FROM_TAG"
-          echo "from_tag=$FROM_TAG" >> $GITHUB_OUTPUT
-      - name: Create Release Notes
-        id: release_notes
-        uses: mikepenz/release-changelog-builder-action@v4
-        with:
-          configuration: .github/release_notes.json
-          toTag: ${{ steps.extract_version.outputs.tag }}
-          fromTag: ${{ steps.extract_version.outputs.from_tag }}
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      - name: Create GH release
-        uses: softprops/action-gh-release@v2
-        with:
-          # Marking betas as pre-releases keeps them from taking the "Latest"
-          # badge on the releases page.
-          prerelease: ${{ steps.extract_version.outputs.prerelease }}
-          make_latest: ${{ steps.extract_version.outputs.prerelease == 'false' }}
-          tag_name: ${{ steps.extract_version.outputs.tag }}
-          token: ${{ secrets.GITHUB_TOKEN }}
-          generate_release_notes: false
-          name: LanceDB v${{ steps.extract_version.outputs.version }}
-          body: ${{ steps.release_notes.outputs.changelog }}

.github/workflows/make-release-commit.yml

@@ -1,15 +1,13 @@
 name: Create release commit
 
-# This workflow increments the version, updates lockfiles, tags the final
-# commit, and pushes it. All SDKs share a single version, so one tag releases
-# all of them.
+# This workflow increments versions, tags the version, and pushes it.
 # When a tag is pushed, another workflow is triggered that creates a GH release
 # and uploads the binaries. This workflow is only for creating the tag.
 
 # This script will enforce that a minor version is incremented if there are any
-# breaking changes since the last minor increment. A breaking change in any SDK
-# bumps the minor version for all of them. If you wish to bypass this check, you
-# can manually increment the version and push the tag.
+# breaking changes since the last minor increment. However, it isn't able to
+# differentiate between breaking changes in Node versus Python. If you wish to
+# bypass this check, you can manually increment the version and push the tag.
 on:
   workflow_dispatch:
     inputs:
@@ -26,6 +24,16 @@ on:
         options:
           - preview
           - stable
+      python:
+        description: 'Make a Python release'
+        required: true
+        default: true
+        type: boolean
+      other:
+        description: 'Make a Node/Rust/Java release'
+        required: true
+        default: true
+        type: boolean
       bump-minor:
         description: 'Bump minor version'
         required: true
@@ -57,17 +65,29 @@ jobs:
         run: |
           git config user.name 'Lance Release'
           git config user.email 'lance-dev@lancedb.com'
-      - name: Bump version
+      - name: Bump Python version
+        if: ${{ inputs.python }}
+        working-directory: python
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          # Need to get the commit before bumping the version, so we can
+          # determine if there are breaking changes in the next step as well.
+          echo "COMMIT_BEFORE_BUMP=$(git rev-parse HEAD)" >> $GITHUB_ENV
+
+          pip install bump-my-version PyGithub packaging
+          bash ../ci/bump_version.sh ${{ inputs.type }} ${{ inputs.bump-minor }} python-v
+      - name: Bump Node/Rust version
+        if: ${{ inputs.other }}
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           pip install bump-my-version PyGithub packaging
-          bash ci/bump_version.sh ${{ inputs.type }} ${{ inputs.bump-minor }}
+          bash ci/bump_version.sh ${{ inputs.type }} ${{ inputs.bump-minor }} v $COMMIT_BEFORE_BUMP
           bash ci/update_lockfiles.sh --amend
-          bash ci/create_release_tag.sh
       - name: Push new version tag
         if: ${{ !inputs.dry_run }}
-        uses: ad-m/github-push-action@881a6320fdb16eb5318c5054f31c218aec2b324c # v1.3.0
+        uses: ad-m/github-push-action@master
         with:
           # Need to use PAT here too to trigger next workflow. See comment above.
           github_token: ${{ secrets.LANCEDB_RELEASE_TOKEN }}

.github/workflows/nodejs.yml

@@ -61,11 +61,6 @@ jobs:
         sudo apt update
         sudo apt install -y protobuf-compiler libssl-dev
     - uses: Swatinem/rust-cache@v2
-      with:
-        # Restore everywhere, but only save from main. Per-PR saves are
-        # unreadable outside their own branch anyway, since GitHub scopes
-        # caches to the creating ref.
-        save-if: ${{ github.ref == 'refs/heads/main' }}
     - name: Format Rust
       run: cargo fmt --all -- --check
     - name: Lint Rust
@@ -108,11 +103,6 @@ jobs:
         cache: 'pnpm'
         cache-dependency-path: nodejs/pnpm-lock.yaml
     - uses: Swatinem/rust-cache@v2
-      with:
-        # Restore everywhere, but only save from main. Per-PR saves are
-        # unreadable outside their own branch anyway, since GitHub scopes
-        # caches to the creating ref.
-        save-if: ${{ github.ref == 'refs/heads/main' }}
     - name: Install dependencies
       run: |
         sudo apt update
@@ -192,11 +182,6 @@ jobs:
         cache-dependency-path: nodejs/pnpm-lock.yaml
     - uses: dtolnay/rust-toolchain@stable
     - uses: Swatinem/rust-cache@v2
-      with:
-        # Restore everywhere, but only save from main. Per-PR saves are
-        # unreadable outside their own branch anyway, since GitHub scopes
-        # caches to the creating ref.
-        save-if: ${{ github.ref == 'refs/heads/main' }}
     - name: Install dependencies
       run: |
         brew install protobuf

.github/workflows/npm-publish.yml

@@ -10,16 +10,10 @@ permissions:
 
 on:
   push:
+    branches:
+      - main
     tags:
       - "v*"
-  # The cross-compiled targets (musl especially) break from toolchain and
-  # dependency changes that nothing else in CI catches, and discovering that
-  # mid-release is expensive. A nightly run keeps that signal while dropping
-  # the full 8-target release matrix from all ~90 pushes to main each month.
-  # `report-failure` files an issue when a nightly breaks.
-  schedule:
-    - cron: "0 8 * * *"
-  workflow_dispatch:
   pull_request:
     # This should trigger a dry run (we skip the final publish step)
     paths:
@@ -32,6 +26,73 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
+  gh-release:
+    if: startsWith(github.ref, 'refs/tags/v')
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+          lfs: true
+      - name: Extract version
+        id: extract_version
+        env:
+          GITHUB_REF: ${{ github.ref }}
+        run: |
+          set -e
+          echo "Extracting tag and version from $GITHUB_REF"
+          if [[ $GITHUB_REF =~ refs/tags/v(.*) ]]; then
+            VERSION=${BASH_REMATCH[1]}
+            TAG=v$VERSION
+            echo "tag=$TAG" >> $GITHUB_OUTPUT
+            echo "version=$VERSION" >> $GITHUB_OUTPUT
+          else
+            echo "Failed to extract version from $GITHUB_REF"
+            exit 1
+          fi
+          echo "Extracted version $VERSION from $GITHUB_REF"
+          if [[ $VERSION =~ beta ]]; then
+            echo "This is a beta release"
+
+            # Get last release (that is not this one)
+            FROM_TAG=$(git tag --sort='version:refname' \
+              | grep ^v \
+              | grep -vF "$TAG" \
+              | python ci/semver_sort.py v \
+              | tail -n 1)
+          else
+            echo "This is a stable release"
+            # Get last stable tag (ignore betas)
+            FROM_TAG=$(git tag --sort='version:refname' \
+              | grep ^v \
+              | grep -vF "$TAG" \
+              | grep -v beta \
+              | python ci/semver_sort.py v \
+              | tail -n 1)
+          fi
+          echo "Found from tag $FROM_TAG"
+          echo "from_tag=$FROM_TAG" >> $GITHUB_OUTPUT
+      - name: Create Release Notes
+        id: release_notes
+        uses: mikepenz/release-changelog-builder-action@v4
+        with:
+          configuration: .github/release_notes.json
+          toTag: ${{ steps.extract_version.outputs.tag }}
+          fromTag: ${{ steps.extract_version.outputs.from_tag }}
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      - name: Create GH release
+        uses: softprops/action-gh-release@v2
+        with:
+          prerelease: ${{ contains('beta', github.ref) }}
+          tag_name: ${{ steps.extract_version.outputs.tag }}
+          token: ${{ secrets.GITHUB_TOKEN }}
+          generate_release_notes: false
+          name: Node/Rust LanceDB v${{ steps.extract_version.outputs.version }}
+          body: ${{ steps.release_notes.outputs.changelog }}
+
   build-lancedb:
     strategy:
       fail-fast: false
@@ -40,18 +101,9 @@ jobs:
           - target: aarch64-apple-darwin
             host: macos-latest
             features: fp16kernels
-            pre_build: |-
-              brew install protobuf
-              # Fat LTO (the workspace default in .cargo/config.toml) is
-              # single-threaded and is the peak-memory step of the build. On
-              # this runner it accounted for ~111 of the job's ~113 minutes,
-              # making it the critical path of the entire publish pipeline.
-              # ThinLTO parallelizes it across the runner's cores, for a few
-              # percent of runtime performance.
-              export CARGO_PROFILE_RELEASE_LTO=thin
-              export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16
+            pre_build: brew install protobuf
           - target: x86_64-pc-windows-msvc
-            host: windows-2025
+            host: windows-latest
             features: ","
             pre_build: |-
               choco install --no-progress protoc ninja nasm
@@ -59,21 +111,12 @@ jobs:
               # There is an issue where choco doesn't add nasm to the path
               export PATH="$PATH:/c/Program Files/NASM"
               nasm -v
-              # See the ThinLTO note on aarch64-apple-darwin above. Keeping
-              # peak memory down is also what lets this run on the standard
-              # 4-core runner: the 8-core larger runner was only needed to
-              # stop fat LTO from OOMing rustc-LLVM.
-              export CARGO_PROFILE_RELEASE_LTO=thin
-              export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16
           - target: aarch64-pc-windows-msvc
-            host: windows-2025
+            host: windows-latest
             features: ","
             pre_build: |-
                 choco install --no-progress protoc
                 rustup target add aarch64-pc-windows-msvc
-                # See the ThinLTO note on aarch64-apple-darwin above.
-                export CARGO_PROFILE_RELEASE_LTO=thin
-                export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16
           - target: x86_64-unknown-linux-gnu
             host: ubuntu-latest
             features: fp16kernels
@@ -146,49 +189,16 @@ jobs:
         with:
           toolchain: stable
           targets: ${{ matrix.settings.target }}
-      # These builds were entirely uncached: the old key was static, so
-      # `actions/cache` (which only writes on a miss) could never refresh it,
-      # and the multi-GB whole-`target/` copy it tried to store never fit the
-      # repo's cache budget, so no entry was ever saved. rust-cache prunes
-      # `target/` to dependency artifacts and keys on Cargo.lock plus the rustc
-      # version, which both fixes the key and keeps entries a sane size.
-      #
-      # This caches dependency *compilation* only. The LTO link of the cdylib
-      # re-runs regardless, since the local crate changes every time, so the
-      # win is larger on the non-LTO jobs than here.
-      - name: Cache cargo (native builds)
-        uses: Swatinem/rust-cache@v2
-        if: ${{ !matrix.settings.docker }}
+      - name: Cache cargo
+        uses: actions/cache@v5
         with:
-          # The release profile and per-target dirs differ from what the test
-          # workflows cache, so these need to be separate entries.
-          key: release-${{ matrix.settings.target }}
-          # Only the nightly run on main writes, so tag and PR runs restore a
-          # warm entry without every dependabot PR writing its own (which would
-          # be unreadable elsewhere anyway, since GitHub scopes caches to the
-          # creating ref). The nightly cadence also keeps entries inside
-          # GitHub's 7-day eviction window, which a tag-only trigger would not.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
-      # Docker builds can use rust-cache too. `target/` already lives on the
-      # host because the whole workspace is bind-mounted into the container, and
-      # rust-cache's prune and save run host-side, so they can manage it -- which
-      # is what keeps the entry to dependency artifacts rather than a multi-GB
-      # copy of everything.
-      #
-      # Two differences from the native builds. The container's CARGO_HOME is
-      # bind-mounted from `.cargo-cache` rather than the host's ~/.cargo, so that
-      # has to be cached explicitly. And the key is derived from the *host* rustc
-      # version, which is not the compiler that produced these artifacts; that is
-      # safe because cargo fingerprints the real compiler and rebuilds on a
-      # mismatch, it just means a base-image toolchain bump costs one cold build
-      # instead of invalidating the key.
-      - name: Cache cargo (docker builds)
-        uses: Swatinem/rust-cache@v2
-        if: ${{ matrix.settings.docker }}
-        with:
-          key: docker-${{ matrix.settings.target }}
-          cache-directories: .cargo-cache
-          save-if: ${{ github.ref == 'refs/heads/main' }}
+          path: |
+            ~/.cargo/registry/index/
+            ~/.cargo/registry/cache/
+            ~/.cargo/git/db/
+            .cargo-cache
+            target/
+          key: nodejs-${{ matrix.settings.target }}-cargo-${{ matrix.settings.host }}
       - name: Install dependencies
         run: pnpm install --frozen-lockfile
       - name: Install Zig
@@ -206,13 +216,9 @@ jobs:
         if: ${{ matrix.settings.docker }}
         with:
           image: ${{ matrix.settings.docker }}
-          # All three mounts must live under `.cargo-cache`, which is what the
-          # cache step above saves. Previously the registry mounts pointed at
-          # `.cargo/...`, a path nothing cached, so the container re-downloaded
-          # the whole crate registry on every run.
           options: "--user 0:0 -v ${{ github.workspace }}/.cargo-cache/git/db:/usr/local/cargo/git/db \
-                    -v ${{ github.workspace }}/.cargo-cache/registry/cache:/usr/local/cargo/registry/cache \
-                    -v ${{ github.workspace }}/.cargo-cache/registry/index:/usr/local/cargo/registry/index \
+                    -v ${{ github.workspace }}/.cargo/registry/cache:/usr/local/cargo/registry/cache \
+                    -v ${{ github.workspace }}/.cargo/registry/index:/usr/local/cargo/registry/index \
                     -v ${{ github.workspace }}:/build -w /build/nodejs"
           run: |
             set -e
@@ -224,16 +230,6 @@ jobs:
               --js ../lancedb/native.js \
               --strip \
               --output-dir dist/
-      # The container runs as root (`--user 0:0`), so everything it wrote to the
-      # mounted cache dirs is root-owned. rust-cache's post step runs as the
-      # runner user and has to both read these and delete from them while
-      # pruning, so hand them back before it runs.
-      - name: Take ownership of docker build output
-        if: ${{ matrix.settings.docker }}
-        run: |
-          sudo chown -R "$(id -u):$(id -g)" \
-            "${{ github.workspace }}/.cargo-cache" \
-            "${{ github.workspace }}/target"
       - name: Build
         run: |
           ${{ matrix.settings.pre_build }}
@@ -247,15 +243,6 @@ jobs:
               --output-dir dist/
         if: ${{ !matrix.settings.docker }}
         shell: bash
-      # The standard Windows runners have ~14 GB free, and a release `target/`
-      # for this workspace is a large fraction of that. Report the remaining
-      # headroom so a build that only just fits is visible before a dependency
-      # bump turns it into a failed release. `always()` so the numbers are
-      # still there when the build is what ran out of space.
-      - name: Report disk headroom
-        if: always()
-        run: df -h
-        shell: bash
       - name: Upload artifact
         uses: actions/upload-artifact@v7
         with:
@@ -406,9 +393,7 @@ jobs:
     name: Report Workflow Failure
     runs-on: ubuntu-latest
     needs: [build-lancedb, test-lancedb, publish]
-    # Nightly runs are the only thing watching the cross-compiled targets now,
-    # so they have to report failures too or the signal is silently lost.
-    if: always() && failure() && (startsWith(github.ref, 'refs/tags/v') || github.event_name == 'schedule')
+    if: always() && failure() && startsWith(github.ref, 'refs/tags/v')
     permissions:
       contents: read
       issues: write

.github/workflows/pypi-publish.yml

@@ -3,7 +3,7 @@ name: PyPI Publish
 on:
   push:
     tags:
-      - 'v*'
+      - 'python-v*'
   pull_request:
     # This should trigger a dry run (we skip the final publish step)
     paths:
@@ -20,15 +20,9 @@ env:
 permissions:
   contents: read
 
-# Without this, a force-push to a PR leaves the previous run going -- including
-# a ~74 minute Windows job and a billed arm64 wheel build.
-concurrency:
-  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
-  cancel-in-progress: true
-
 jobs:
   linux:
-    name: Python ${{ matrix.config.package_name }} ${{ matrix.config.platform }} manylinux${{ matrix.config.manylinux }}
+    name: Python ${{ matrix.config.platform }} manylinux${{ matrix.config.manylinux }}
     timeout-minutes: 60
     strategy:
       matrix:
@@ -37,28 +31,11 @@ jobs:
             manylinux: "2_28"
             extra_args: "--features fp16kernels"
             runner: ubuntu-22.04
-            package_name: "lancedb"
-            rustflags: ""
           # For successful fat LTO builds, we need a large runner to avoid OOM errors.
           - platform: aarch64
             manylinux: "2_28"
             extra_args: "--features fp16kernels"
             runner: ubuntu-2404-8x-arm64
-            package_name: "lancedb"
-            rustflags: ""
-          # `lancedb-compat`: pre-Haswell-friendly variant for x86_64 hosts
-          # without AVX2 (Sandy Bridge / Ivy Bridge / Westmere on Intel,
-          # Bulldozer / Piledriver / Steamroller on AMD). Compiled at the
-          # `x86-64-v2` baseline; runtime SIMD dispatch in lance-linalg
-          # picks the appropriate tier (scalar / AVX / AVX+FMA / AVX2+FMA
-          # / AVX-512) at load time. Same import as `lancedb` -- conflicts
-          # at install time, so users pick one.
-          - platform: x86_64
-            manylinux: "2_28"
-            extra_args: ""
-            runner: ubuntu-22.04
-            package_name: "lancedb-compat"
-            rustflags: "-Ctarget-cpu=x86-64-v2"
     runs-on: ${{ matrix.config.runner }}
     steps:
       - uses: actions/checkout@v6
@@ -75,13 +52,11 @@ jobs:
           args: "--release --strip ${{ matrix.config.extra_args }}"
           arm-build: ${{ matrix.config.platform == 'aarch64' }}
           manylinux: ${{ matrix.config.manylinux }}
-          package-name: ${{ matrix.config.package_name }}
-          rustflags: ${{ matrix.config.rustflags }}
       - uses: actions/upload-artifact@v7
-        if: startsWith(github.ref, 'refs/tags/v')
+        if: startsWith(github.ref, 'refs/tags/python-v')
         with:
-          name: wheels-linux-${{ matrix.config.package_name }}-${{ matrix.config.platform }}-${{ matrix.config.manylinux }}
-          path: target/wheels/*.whl
+          name: wheels-linux-${{ matrix.config.platform }}-${{ matrix.config.manylinux }}
+          path: target/wheels/lancedb-*.whl
           if-no-files-found: error
   mac:
     timeout-minutes: 90
@@ -107,7 +82,7 @@ jobs:
           python-minor-version: 10
           args: "--release --strip --target ${{ matrix.config.target }} --features fp16kernels"
       - uses: actions/upload-artifact@v7
-        if: startsWith(github.ref, 'refs/tags/v')
+        if: startsWith(github.ref, 'refs/tags/python-v')
         with:
           name: wheels-mac-${{ matrix.config.target }}
           path: target/wheels/lancedb-*.whl
@@ -128,26 +103,19 @@ jobs:
         uses: actions/setup-python@v6
         with:
           python-version: "3.13"
-      # NOTE: caching cargo here would be a no-op. This workflow only runs on
-      # tags and PRs, and GitHub only lets a run restore caches from its own ref
-      # or the default branch -- so with no run on main there is nothing that
-      # can populate an entry the release build would be allowed to read. Fixing
-      # this needs a main/nightly trigger (which would also catch wheel-build
-      # breakage before a release); the ~74 minutes here is otherwise dominated
-      # by the fat-LTO link, which no cache avoids.
       - uses: ./.github/workflows/build_windows_wheel
         with:
           python-minor-version: 10
           args: "--release --strip"
       - uses: actions/upload-artifact@v7
-        if: startsWith(github.ref, 'refs/tags/v')
+        if: startsWith(github.ref, 'refs/tags/python-v')
         with:
           name: wheels-windows
           path: target/wheels/lancedb-*.whl
           if-no-files-found: error
   publish:
     name: Publish wheels
-    if: startsWith(github.ref, 'refs/tags/v')
+    if: startsWith(github.ref, 'refs/tags/python-v')
     needs: [linux, mac, windows]
     runs-on: ubuntu-latest
     permissions:
@@ -177,7 +145,7 @@ jobs:
           FURY_TOKEN: ${{ secrets.FURY_TOKEN }}
         run: |
           shopt -s nullglob
-          WHEELS=(target/wheels/*.whl)
+          WHEELS=(target/wheels/lancedb-*.whl)
           if [[ ${#WHEELS[@]} -eq 0 ]]; then
             echo "No wheels found in target/wheels/" >&2
             exit 1
@@ -196,6 +164,72 @@ jobs:
         uses: pypa/gh-action-pypi-publish@release/v1
         with:
           packages-dir: target/wheels/
+  gh-release:
+    if: startsWith(github.ref, 'refs/tags/python-v')
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v6
+        with:
+          fetch-depth: 0
+          lfs: true
+      - name: Extract version
+        id: extract_version
+        env:
+          GITHUB_REF: ${{ github.ref }}
+        run: |
+          set -e
+          echo "Extracting tag and version from $GITHUB_REF"
+          if [[ $GITHUB_REF =~ refs/tags/python-v(.*) ]]; then
+            VERSION=${BASH_REMATCH[1]}
+            TAG=python-v$VERSION
+            echo "tag=$TAG" >> $GITHUB_OUTPUT
+            echo "version=$VERSION" >> $GITHUB_OUTPUT
+          else
+            echo "Failed to extract version from $GITHUB_REF"
+            exit 1
+          fi
+          echo "Extracted version $VERSION from $GITHUB_REF"
+          if [[ $VERSION =~ beta ]]; then
+            echo "This is a beta release"
+
+            # Get last release (that is not this one)
+            FROM_TAG=$(git tag --sort='version:refname' \
+              | grep ^python-v \
+              | grep -vF "$TAG" \
+              | python ci/semver_sort.py python-v \
+              | tail -n 1)
+          else
+            echo "This is a stable release"
+            # Get last stable tag (ignore betas)
+            FROM_TAG=$(git tag --sort='version:refname' \
+              | grep ^python-v \
+              | grep -vF "$TAG" \
+              | grep -v beta \
+              | python ci/semver_sort.py python-v \
+              | tail -n 1)
+          fi
+          echo "Found from tag $FROM_TAG"
+          echo "from_tag=$FROM_TAG" >> $GITHUB_OUTPUT
+      - name: Create Python Release Notes
+        id: python_release_notes
+        uses: mikepenz/release-changelog-builder-action@v4
+        with:
+          configuration: .github/release_notes.json
+          toTag: ${{ steps.extract_version.outputs.tag }}
+          fromTag: ${{ steps.extract_version.outputs.from_tag }}
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      - name: Create Python GH release
+        uses: softprops/action-gh-release@v2
+        with:
+          prerelease: ${{ contains('beta', github.ref) }}
+          tag_name: ${{ steps.extract_version.outputs.tag }}
+          token: ${{ secrets.GITHUB_TOKEN }}
+          generate_release_notes: false
+          name: Python LanceDB v${{ steps.extract_version.outputs.version }}
+          body: ${{ steps.python_release_notes.outputs.changelog }}
   report-failure:
     name: Report Workflow Failure
     runs-on: ubuntu-latest
@@ -203,7 +237,7 @@ jobs:
     permissions:
       contents: read
       issues: write
-    if: always() && failure() && startsWith(github.ref, 'refs/tags/v')
+    if: always() && failure() && startsWith(github.ref, 'refs/tags/python-v')
     steps:
       - uses: actions/checkout@v6
       - uses: ./.github/actions/create-failure-issue

.github/workflows/python.yml

@@ -108,15 +108,6 @@ jobs:
         run: |
           sudo apt update
           sudo apt install -y protobuf-compiler
-      # `pip install -e .` builds the extension with maturin, which is most of
-      # this job's ~33 minutes. It had no Rust cache, so every dependency was
-      # recompiled from scratch on every run.
-      - uses: Swatinem/rust-cache@v2
-        with:
-          # Restore everywhere, but only save from main. Per-PR saves are
-          # unreadable outside their own branch anyway, since GitHub scopes
-          # caches to the creating ref.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
       - name: Install
         run: |
           pip install --extra-index-url https://pypi.fury.io/lance-format/ --extra-index-url https://pypi.fury.io/lancedb/ -e .[tests,dev,embeddings]
@@ -177,14 +168,6 @@ jobs:
         uses: actions/setup-python@v6
         with:
           python-version: "3.13"
-      # maturin runs cargo natively on macOS (docker is Linux-only), so the host
-      # target dir is cacheable. This job had no Rust cache.
-      - uses: Swatinem/rust-cache@v2
-        with:
-          # Restore everywhere, but only save from main. Per-PR saves are
-          # unreadable outside their own branch anyway, since GitHub scopes
-          # caches to the creating ref.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
       - uses: ./.github/workflows/build_mac_wheel
         with:
           args: --profile ci
@@ -214,14 +197,6 @@ jobs:
         uses: actions/setup-python@v6
         with:
           python-version: "3.13"
-      # maturin runs cargo natively on Windows (docker is Linux-only), so the
-      # host target dir is cacheable. This job had no Rust cache at all and so
-      # rebuilt every dependency from scratch on every run.
-      - uses: Swatinem/rust-cache@v2
-        with:
-          # Restore everywhere, but only save from main. The repo sits at
-          # GitHub's cache cap, so per-PR saves just evict main's entries.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
       - uses: ./.github/workflows/build_windows_wheel
         with:
           args: --profile ci
@@ -249,14 +224,6 @@ jobs:
         uses: actions/setup-python@v6
         with:
           python-version: "3.10"
-      # As with Doctest, `pip install -e .` compiles the extension and this job
-      # had no Rust cache, which is most of its ~37 minutes.
-      - uses: Swatinem/rust-cache@v2
-        with:
-          # Restore everywhere, but only save from main. Per-PR saves are
-          # unreadable outside their own branch anyway, since GitHub scopes
-          # caches to the creating ref.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
       - name: Install lancedb
         run: |
           pip install "pydantic<2"

.github/workflows/rust.yml

@@ -48,11 +48,6 @@ jobs:
         with:
           components: rustfmt, clippy
       - uses: Swatinem/rust-cache@v2
-        with:
-          # Restore everywhere, but only save from main. Per-PR saves are
-          # unreadable outside their own branch anyway, since GitHub scopes
-          # caches to the creating ref.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
       - name: Install dependencies
         run: |
           sudo apt update
@@ -94,11 +89,6 @@ jobs:
         run: rm -f Cargo.lock
       - uses: rui314/setup-mold@v1
       - uses: Swatinem/rust-cache@v2
-        with:
-          # Restore everywhere, but only save from main. Per-PR saves are
-          # unreadable outside their own branch anyway, since GitHub scopes
-          # caches to the creating ref.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
       - name: Install dependencies
         run: |
           sudo apt update
@@ -108,7 +98,7 @@ jobs:
           cargo build --profile ci --benches --all-features --tests
 
   linux:
-    timeout-minutes: 60
+    timeout-minutes: 30
     # To build all features, we need more disk space than is available
     # on the free OSS github runner. This is mostly due to the the
     # sentence-transformers feature.
@@ -128,11 +118,6 @@ jobs:
           fetch-depth: 0
           lfs: true
       - uses: Swatinem/rust-cache@v2
-        with:
-          # Restore everywhere, but only save from main. Per-PR saves are
-          # unreadable outside their own branch anyway, since GitHub scopes
-          # caches to the creating ref.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
       - name: Install dependencies
         run: |
           sudo apt update
@@ -140,26 +125,10 @@ jobs:
       - uses: rui314/setup-mold@v1
       - name: Make Swap
         run: |
-          swapfile=/swapfile
-          min_swap_bytes=$((15 * 1024 * 1024 * 1024))
-          active_swap_bytes="$(sudo swapon --show=NAME,SIZE --bytes --noheadings | awk '$1 == "/swapfile" { print $2 }')"
-          if [ -n "$active_swap_bytes" ]; then
-            if [ "$active_swap_bytes" -ge "$min_swap_bytes" ]; then
-              echo "/swapfile is already active with enough space; skipping swap creation"
-              exit 0
-            fi
-            echo "/swapfile is already active but smaller than 16G; using /mnt/lancedb-swapfile"
-            swapfile=/mnt/lancedb-swapfile
-          fi
-          if sudo swapon --show=NAME --noheadings | grep -Fxq "$swapfile"; then
-            echo "$swapfile is already active; skipping swap creation"
-            exit 0
-          fi
-          sudo rm -f "$swapfile"
-          sudo fallocate -l 16G "$swapfile"
-          sudo chmod 600 "$swapfile"
-          sudo mkswap "$swapfile"
-          sudo swapon "$swapfile"
+          sudo fallocate -l 16G /swapfile
+          sudo chmod 600 /swapfile
+          sudo mkswap /swapfile
+          sudo swapon /swapfile
       - name: Build
         run: cargo build --profile ci --all-features --tests --locked --examples
       - name: Run feature tests
@@ -173,7 +142,7 @@ jobs:
         run: CARGO_ARGS="--profile ci" make -C ./lancedb remote-tests
 
   macos:
-    timeout-minutes: 60
+    timeout-minutes: 30
     strategy:
       matrix:
         mac-runner: ["macos-14", "macos-15"]
@@ -190,11 +159,6 @@ jobs:
       - name: CPU features
         run: sysctl -a | grep cpu
       - uses: Swatinem/rust-cache@v2
-        with:
-          # Restore everywhere, but only save from main. Per-PR saves are
-          # unreadable outside their own branch anyway, since GitHub scopes
-          # caches to the creating ref.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
       - name: Install dependencies
         run: brew install protobuf
       - name: Run tests
@@ -207,19 +171,12 @@ jobs:
           cargo test --profile ci --features $ALL_FEATURES --locked
 
   windows:
+    runs-on: windows-2022
     strategy:
-      fail-fast: false
       matrix:
-        include:
-          - target: x86_64-pc-windows-msvc
-            runner: windows-2022
-          # windows-11-arm is a standard runner, so it is free on public repos.
-          # Running natively lets the aarch64 tests actually execute -- this
-          # job used to cross-compile them and then skip the test step, paying
-          # full codegen and link cost for a compile check.
-          - target: aarch64-pc-windows-msvc
-            runner: windows-11-arm
-    runs-on: ${{ matrix.runner }}
+        target:
+          - x86_64-pc-windows-msvc
+          - aarch64-pc-windows-msvc
     defaults:
       run:
         working-directory: rust/lancedb
@@ -228,11 +185,6 @@ jobs:
       - name: Set target
         run: rustup target add ${{ matrix.target }}
       - uses: Swatinem/rust-cache@v2
-        with:
-          # Restore everywhere, but only save from main. Per-PR saves are
-          # unreadable outside their own branch anyway, since GitHub scopes
-          # caches to the creating ref.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
       - name: Install Protoc v21.12
         run: choco install --no-progress protoc
       - name: Build
@@ -240,12 +192,11 @@ jobs:
           $env:VCPKG_ROOT = $env:VCPKG_INSTALLATION_ROOT
           cargo build --profile ci --features aws,remote --tests --locked --target ${{ matrix.target }}
       - name: Run tests
+        # Can only run tests when target matches host
+        if: ${{ matrix.target == 'x86_64-pc-windows-msvc' }}
         run: |
           $env:VCPKG_ROOT = $env:VCPKG_INSTALLATION_ROOT
-          # `--target` has to match the build step above. Without it cargo uses
-          # target/ci/ rather than target/<triple>/ci/ and rebuilds the entire
-          # dependency graph a second time.
-          cargo test --profile ci --features aws,remote --locked --target ${{ matrix.target }}
+          cargo test --profile ci --features aws,remote --locked
 
   msrv:
     # Check the minimum supported Rust version
@@ -271,11 +222,6 @@ jobs:
         with:
           toolchain: ${{ matrix.msrv }}
       - uses: Swatinem/rust-cache@v2
-        with:
-          # Restore everywhere, but only save from main. Per-PR saves are
-          # unreadable outside their own branch anyway, since GitHub scopes
-          # caches to the creating ref.
-          save-if: ${{ github.ref == 'refs/heads/main' }}
       - name: Downgrade  dependencies
         # These packages have newer requirements for MSRV
         run: |
@@ -296,18 +242,16 @@ jobs:
           cargo update -p aws-types --precise 1.3.9
           cargo update -p aws-sigv4 --precise 1.3.5
           cargo update -p aws-credential-types --precise 1.2.8
-          # aws-smithy-checksums must stay at or above 0.63.13: OpenDAL's S3
-          # service needs crc-fast ~1.9, and older releases pin it to ~1.3.
-          cargo update -p aws-smithy-checksums --precise 0.63.13
+          cargo update -p aws-smithy-checksums --precise 0.63.9
           cargo update -p aws-smithy-runtime --precise 1.9.3
-          cargo update -p aws-smithy-http --precise 0.62.6
-          cargo update -p aws-smithy-eventstream --precise 0.60.14
+          cargo update -p aws-smithy-http --precise 0.62.4
+          cargo update -p aws-smithy-eventstream --precise 0.60.12
           cargo update -p aws-smithy-http-client --precise 1.1.3
           cargo update -p aws-smithy-observability --precise 0.1.4
           cargo update -p aws-smithy-query --precise 0.60.8
-          cargo update -p aws-smithy-runtime-api --precise 1.9.3
-          cargo update -p aws-smithy-async --precise 1.2.7
-          cargo update -p aws-smithy-types --precise 1.3.6
+          cargo update -p aws-smithy-runtime-api --precise 1.9.1
+          cargo update -p aws-smithy-async --precise 1.2.6
+          cargo update -p aws-smithy-types --precise 1.3.5
           cargo update -p aws-smithy-xml --precise 0.60.11
           cargo update -p home --precise 0.5.9
       - name: cargo +${{ matrix.msrv }} check

AGENTS.md

@@ -92,8 +92,6 @@ Python bindings changes:
     * Should use `LOOP.run()` to call the corresponding `AsyncTable` method.
 6. Add concrete sync method to `RemoteTable` class in `python/python/lancedb/remote/table.py`.
 7. Add unit test in `python/tests/test_table.py`.
-8. If you added a new public class or module-level function (not just a method on an
-   existing class), expose it in the API reference. See "Python API reference" below.
 
 TypeScript bindings changes:
 
@@ -105,33 +103,6 @@ TypeScript bindings changes:
 5. Add test in `nodejs/__test__/table.test.ts`.
 6. Run `npm run docs` to generate TypeScript documentation.
 
-## Python API reference
-
-`docs/src/python/python.md` is the entire Python API reference. It is maintained by
-hand, and anything not listed there is not rendered at all, so new public classes and
-module-level functions have to be added explicitly. How depends on the module:
-
-* `lancedb.index`, `lancedb.embeddings`, `lancedb.remote`, and `lancedb.rerankers` are
-  rendered by a single directive each, driven by the module's `__all__`. Add the new
-  name to `__all__` and it appears; forget, and it is silently omitted.
-* Everything else (`lancedb`, `lancedb.table`, `lancedb.query`, `lancedb.db`, ...) is
-  listed symbol by symbol. Add a `::: lancedb.<module>.<Name>` line to the matching
-  section, and remember that the page separates synchronous and asynchronous APIs.
-
-Deliberately undocumented: concrete implementations reached through an abstract base
-(`LanceTable`, `LanceDBConnection`, `RemoteDBConnection`), query base classes already
-covered by `inherited_members`, and internal helpers.
-
-Cross-references in docstrings use mkdocstrings syntax, `[text][lancedb.table.Table]`.
-Plain relative links such as `[Table](Table)` do not resolve. To check your work:
-
-```shell
-pip install -r docs/requirements.txt
-cd docs && PYTHONPATH=. mkdocs build
-```
-
-The docs site only builds on pushes to `main`, so this is not covered by PR CI.
-
 ## Review Guidelines
 
 Please consider the following when reviewing code contributions.

Cargo.toml

@@ -13,20 +13,20 @@ categories = ["database-implementations"]
 rust-version = "1.91.0"
 
 [workspace.dependencies]
-lance = { "version" = "=10.0.0", default-features = false }
-lance-core = "=10.0.0"
-lance-datagen = "=10.0.0"
-lance-file = "=10.0.0"
-lance-io = { "version" = "=10.0.0", default-features = false }
-lance-index = "=10.0.0"
-lance-linalg = "=10.0.0"
-lance-namespace = "=10.0.0"
-lance-namespace-impls = { "version" = "=10.0.0", default-features = false }
-lance-table = "=10.0.0"
-lance-testing = "=10.0.0"
-lance-datafusion = "=10.0.0"
-lance-encoding = "=10.0.0"
-lance-arrow = "=10.0.0"
+lance = { "version" = "=8.0.0", default-features = false }
+lance-core = "=8.0.0"
+lance-datagen = "=8.0.0"
+lance-file = "=8.0.0"
+lance-io = { "version" = "=8.0.0", default-features = false }
+lance-index = "=8.0.0"
+lance-linalg = "=8.0.0"
+lance-namespace = "=8.0.0"
+lance-namespace-impls = { "version" = "=8.0.0", default-features = false }
+lance-table = "=8.0.0"
+lance-testing = "=8.0.0"
+lance-datafusion = "=8.0.0"
+lance-encoding = "=8.0.0"
+lance-arrow = "=8.0.0"
 ahash = "0.8"
 # Note that this one does not include pyarrow
 arrow = { version = "58.0.0", optional = false }
@@ -39,23 +39,21 @@ arrow-schema = "58.0.0"
 arrow-select = "58.0.0"
 arrow-cast = "58.0.0"
 async-trait = "0"
-datafusion = { version = "54.0.0", default-features = false }
-datafusion-catalog = "54.0.0"
-datafusion-common = { version = "54.0.0", default-features = false }
-datafusion-execution = "54.0.0"
-datafusion-expr = "54.0.0"
-datafusion-functions = "54.0.0"
-datafusion-physical-plan = "54.0.0"
-datafusion-physical-expr = "54.0.0"
-datafusion-sql = "54.0.0"
+datafusion = { version = "53.0.0", default-features = false }
+datafusion-catalog = "53.0.0"
+datafusion-common = { version = "53.0.0", default-features = false }
+datafusion-execution = "53.0.0"
+datafusion-expr = "53.0.0"
+datafusion-functions = "53.0.0"
+datafusion-physical-plan = "53.0.0"
+datafusion-physical-expr = "53.0.0"
+datafusion-sql = "53.0.0"
 env_logger = "0.11"
 half = { "version" = "2.7.1", default-features = false, features = [
     "num-traits",
 ] }
-futures = "0.3"
+futures = "0"
 log = "0.4"
-metrics = "0.24"
-metrics-util = "0.19"
 moka = { version = "0.12", features = ["future"] }
 object_store = "0.13.2"
 pin-project = "1.0.7"
@@ -64,6 +62,7 @@ snafu = "0.8"
 url = "2"
 num-traits = "0.2"
 regex = "1.10"
+lazy_static = "1"
 semver = "1.0.25"
 chrono = "0.4"
 

ci/bump_version.sh

@@ -2,15 +2,20 @@ set -e
 
 RELEASE_TYPE=${1:-"stable"}
 BUMP_MINOR=${2:-false}
-HEAD_SHA=$(git rev-parse HEAD)
+TAG_PREFIX=${3:-"v"} # Such as "python-v"
+HEAD_SHA=${4:-$(git rev-parse HEAD)}
 
-readonly TAG_PREFIX="v"
 readonly SELF_DIR=$(cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
-readonly BUMP_ARGS="--no-tag"
 
 PREV_TAG=$(git tag --sort='version:refname' | grep ^$TAG_PREFIX | python $SELF_DIR/semver_sort.py $TAG_PREFIX | tail -n 1)
 echo "Found previous tag $PREV_TAG"
 
+# Initially, we don't want to tag if we are doing stable, because we will bump
+# again later. See comment at end for why.
+if [[ "$RELEASE_TYPE" == 'stable' ]]; then 
+  BUMP_ARGS="--no-tag"
+fi
+
 # If last is stable and not bumping minor
 if [[ $PREV_TAG != *beta* ]]; then
     if [[ "$BUMP_MINOR" != "false" ]]; then
@@ -34,12 +39,12 @@ fi
 # a stable version, bump the pre-release level ("pre_l") to make it stable.
 if [[ $RELEASE_TYPE == 'stable' ]]; then
   # X.Y.Z-beta.N -> X.Y.Z
-  bump-my-version bump -vv $BUMP_ARGS pre_l
+  bump-my-version bump -vv pre_l
 fi
 
 # Validate that we have incremented version appropriately for breaking changes
-NEW_VERSION=$(python -c 'import tomllib; print(tomllib.load(open(".bumpversion.toml", "rb"))["tool"]["bumpversion"]["current_version"])')
-NEW_TAG="$TAG_PREFIX$NEW_VERSION"
+NEW_TAG=$(git describe --tags --exact-match HEAD)
+NEW_VERSION=$(echo $NEW_TAG | sed "s/^$TAG_PREFIX//")
 LAST_STABLE_RELEASE=$(git tag --sort='version:refname' | grep ^$TAG_PREFIX | grep -v beta | grep -vF "$NEW_TAG" | python $SELF_DIR/semver_sort.py $TAG_PREFIX | tail -n 1)
 LAST_STABLE_VERSION=$(echo $LAST_STABLE_RELEASE | sed "s/^$TAG_PREFIX//")
 

ci/create_release_tag.sh

@@ -1,21 +0,0 @@
-#!/usr/bin/env bash
-set -euo pipefail
-
-RELEASE_VERSION=$(python -c 'import tomllib; print(tomllib.load(open(".bumpversion.toml", "rb"))["tool"]["bumpversion"]["current_version"])')
-RELEASE_TAG="v${RELEASE_VERSION}"
-
-if git rev-parse --quiet --verify "refs/tags/${RELEASE_TAG}" >/dev/null; then
-  echo "Release tag ${RELEASE_TAG} already exists" >&2
-  exit 1
-fi
-
-git tag --annotate "$RELEASE_TAG" --message "Release ${RELEASE_TAG}"
-
-HEAD_SHA=$(git rev-parse HEAD)
-TAG_SHA=$(git rev-parse "refs/tags/${RELEASE_TAG}^{}")
-if [[ "$TAG_SHA" != "$HEAD_SHA" ]]; then
-  echo "Release tag ${RELEASE_TAG} points to ${TAG_SHA}, expected ${HEAD_SHA}" >&2
-  exit 1
-fi
-
-echo "Created ${RELEASE_TAG} at ${HEAD_SHA}"

ci/test_validate_stable_lance.py

@@ -1,72 +0,0 @@
-import tempfile
-import unittest
-from pathlib import Path
-
-from validate_stable_lance import validate
-
-
-class ValidateStableLanceTest(unittest.TestCase):
-    def write_fixture(
-        self,
-        root: Path,
-        *,
-        rust: str = "=10.0.0",
-        python: str = "10.0.0",
-        java: str = "10.0.0",
-    ) -> None:
-        (root / "python").mkdir()
-        (root / "java").mkdir()
-        (root / "Cargo.toml").write_text(
-            f'[workspace.dependencies]\nlance = "{rust}"\nlance-core = "{rust}"\n'
-        )
-        (root / "python" / "pyproject.toml").write_text(
-            '[project.optional-dependencies]\ntests = ["pylance==' + python + '"]\n'
-        )
-        (root / "java" / "pom.xml").write_text(
-            "<project><properties><lance-core.version>"
-            + java
-            + "</lance-core.version></properties></project>"
-        )
-
-    def test_accepts_matching_stable_versions(self) -> None:
-        with tempfile.TemporaryDirectory() as temp_dir:
-            root = Path(temp_dir)
-            self.write_fixture(root)
-            self.assertEqual(validate(root), "10.0.0")
-
-    def test_rejects_prerelease(self) -> None:
-        with tempfile.TemporaryDirectory() as temp_dir:
-            root = Path(temp_dir)
-            self.write_fixture(root, python="10.0.0rc1")
-            with self.assertRaisesRegex(ValueError, "not stable"):
-                validate(root)
-
-    def test_rejects_sdk_mismatch(self) -> None:
-        with tempfile.TemporaryDirectory() as temp_dir:
-            root = Path(temp_dir)
-            self.write_fixture(root, java="9.0.0")
-            with self.assertRaisesRegex(ValueError, "do not match across SDKs"):
-                validate(root)
-
-    def test_rejects_non_exact_rust_dependency(self) -> None:
-        with tempfile.TemporaryDirectory() as temp_dir:
-            root = Path(temp_dir)
-            self.write_fixture(root, rust="10.0.0")
-            with self.assertRaisesRegex(ValueError, "not exact"):
-                validate(root)
-
-    def test_rejects_unpublished_rust_source(self) -> None:
-        with tempfile.TemporaryDirectory() as temp_dir:
-            root = Path(temp_dir)
-            self.write_fixture(root)
-            (root / "Cargo.toml").write_text(
-                "[workspace.dependencies]\n"
-                'lance = { version = "=10.0.0", git = "https://example.com/lance", '
-                'tag = "v10.0.0" }\n'
-            )
-            with self.assertRaisesRegex(ValueError, "unpublished source fields"):
-                validate(root)
-
-
-if __name__ == "__main__":
-    unittest.main()

ci/validate_stable_lance.py

@@ -1,112 +1,34 @@
-#!/usr/bin/env python3
-"""Validate that every SDK uses the same published stable Lance release."""
-
-from __future__ import annotations
-
-import re
-import xml.etree.ElementTree as ET
-from pathlib import Path
-
 import tomllib
 
-STABLE_VERSION = re.compile(r"[0-9]+\.[0-9]+\.[0-9]+")
+found_preview_lance = False
 
+with open("Cargo.toml", "rb") as f:
+    cargo_data = tomllib.load(f)
 
-def _stable_version(raw: str, *, dependency: str, exact: bool = False) -> str:
-    value = raw.strip()
-    if exact and not value.startswith("="):
-        raise ValueError(f"Dependency '{dependency}' is not exact: {raw}")
-    value = value.removeprefix("=").removeprefix("v")
-    if STABLE_VERSION.fullmatch(value) is None:
-        raise ValueError(f"Dependency '{dependency}' is not stable: {raw}")
-    return value
+    for name, dep in cargo_data["workspace"]["dependencies"].items():
+        if name == "lance" or name.startswith("lance-"):
+            if isinstance(dep, str):
+                version = dep
+            elif isinstance(dep, dict):
+                # Version doesn't have the beta tag in it, so we instead look
+                # at the git tag.
+                version = dep.get('tag', dep.get('version'))
+            else:
+                raise ValueError("Unexpected type for dependency: " + str(dep))
 
+            if "beta" in version:
+                found_preview_lance = True
+                print(f"Dependency '{name}' is a preview version: {version}")
 
-def rust_lance_version(repo_root: Path) -> str:
-    with (repo_root / "Cargo.toml").open("rb") as cargo_file:
-        dependencies = tomllib.load(cargo_file)["workspace"]["dependencies"]
+with open("python/pyproject.toml", "rb") as f:
+    py_proj_data = tomllib.load(f)
 
-    versions: dict[str, str] = {}
-    for name, dependency in dependencies.items():
-        if name != "lance" and not name.startswith("lance-"):
-            continue
+    for dep in py_proj_data["project"]["dependencies"]:
+        if dep.startswith("pylance"):
+            if "b" in dep:
+                found_preview_lance = True
+                print(f"Dependency '{dep}' is a preview version")
+            break  # Only one pylance dependency
 
-        if isinstance(dependency, str):
-            raw_version = dependency
-        elif isinstance(dependency, dict):
-            forbidden_sources = [
-                source
-                for source in ("git", "path", "branch", "rev", "tag")
-                if source in dependency
-            ]
-            if forbidden_sources:
-                joined = ", ".join(forbidden_sources)
-                raise ValueError(
-                    f"Dependency '{name}' uses unpublished source fields: {joined}"
-                )
-            raw_version = dependency.get("version")
-            if raw_version is None:
-                raise ValueError(f"Dependency '{name}' has no version")
-        else:
-            raise TypeError(f"Dependency '{name}' has an unexpected definition")
-
-        versions[name] = _stable_version(raw_version, dependency=name, exact=True)
-
-    if not versions:
-        raise ValueError("No Rust Lance dependencies found")
-    unique_versions = set(versions.values())
-    if len(unique_versions) != 1:
-        details = ", ".join(f"{name}={version}" for name, version in versions.items())
-        raise ValueError(f"Rust Lance dependency versions do not match: {details}")
-    return unique_versions.pop()
-
-
-def python_lance_version(repo_root: Path) -> str:
-    with (repo_root / "python" / "pyproject.toml").open("rb") as pyproject_file:
-        pyproject = tomllib.load(pyproject_file)
-
-    requirements = pyproject["project"]["optional-dependencies"]["tests"]
-    pylance_requirements = [
-        requirement for requirement in requirements if requirement.startswith("pylance")
-    ]
-    if len(pylance_requirements) != 1:
-        raise ValueError(
-            "Expected exactly one pylance requirement in the Python test dependencies"
-        )
-
-    requirement = pylance_requirements[0]
-    prefix = "pylance=="
-    if not requirement.startswith(prefix):
-        raise ValueError(f"Python test dependency is not exact: {requirement}")
-    return _stable_version(requirement[len(prefix) :], dependency="pylance")
-
-
-def java_lance_version(repo_root: Path) -> str:
-    pom_root = ET.parse(repo_root / "java" / "pom.xml").getroot()
-    versions = [
-        element.text
-        for element in pom_root.iter()
-        if element.tag.rsplit("}", 1)[-1] == "lance-core.version"
-    ]
-    if len(versions) != 1 or versions[0] is None:
-        raise ValueError("Expected exactly one Java lance-core.version property")
-    return _stable_version(versions[0], dependency="Java lance-core")
-
-
-def validate(repo_root: Path) -> str:
-    versions = {
-        "Rust": rust_lance_version(repo_root),
-        "Python": python_lance_version(repo_root),
-        "Java": java_lance_version(repo_root),
-    }
-    if len(set(versions.values())) != 1:
-        details = ", ".join(f"{sdk}={version}" for sdk, version in versions.items())
-        raise ValueError(
-            f"Lance dependency versions do not match across SDKs: {details}"
-        )
-    return versions["Rust"]
-
-
-if __name__ == "__main__":
-    version = validate(Path(__file__).resolve().parents[1])
-    print(f"Validated published stable Lance v{version} across Rust, Python, and Java")
+if found_preview_lance:
+    raise ValueError("Found preview version of Lance in dependencies")

deny.toml

@@ -51,6 +51,18 @@ ignore = [
     # https://rustsec.org/advisories/RUSTSEC-2024-0436
     { id = "RUSTSEC-2024-0436", reason = "transitive via datafusion; awaiting ecosystem migration" },
 
+    # encoding: unmaintained. Reached through lindera-dictionary, which is
+    # required by the native Lindera tokenizer path. Lindera has not migrated
+    # off this crate yet.
+    # https://rustsec.org/advisories/RUSTSEC-2021-0153
+    { id = "RUSTSEC-2021-0153", reason = "transitive via lindera-dictionary for native Lindera tokenizer" },
+
+    # fast-float: unsound and unmaintained. Reached only through polars-arrow
+    # from the optional Polars integration; replacement requires a Polars
+    # dependency upgrade.
+    # https://rustsec.org/advisories/RUSTSEC-2024-0379
+    { id = "RUSTSEC-2024-0379", reason = "transitive via polars-arrow; waiting on Polars migration" },
+
     # tantivy: segfault on malformed input due to missing bounds check.
     # Pulled in via lance for full-text search. We only feed tantivy
     # documents we construct ourselves, not attacker-controlled bytes.
@@ -68,6 +80,18 @@ ignore = [
     # https://rustsec.org/advisories/RUSTSEC-2025-0119
     { id = "RUSTSEC-2025-0119", reason = "transitive via hf-hub/indicatif; cosmetic formatting crate" },
 
+    # bincode: unmaintained. Reached through lindera and lindera-dictionary,
+    # which are required by the native Lindera tokenizer path. Lindera has not
+    # migrated to another serialization format yet.
+    # https://rustsec.org/advisories/RUSTSEC-2025-0141
+    { id = "RUSTSEC-2025-0141", reason = "transitive via lindera/lindera-dictionary for native Lindera tokenizer" },
+
+    # lru: soundness issue in IterMut. Reached only through aws-sdk-s3 in
+    # LanceDB's dev-dependency graph; LanceDB does not use that iterator
+    # directly. Clearing this requires the AWS SDK chain to update lru.
+    # https://rustsec.org/advisories/RUSTSEC-2026-0002
+    { id = "RUSTSEC-2026-0002", reason = "transitive via aws-sdk-s3 dev-dependency; waiting on AWS SDK lru upgrade" },
+
     # rustls-webpki 0.101.7 (old major line): name-constraint checks for
     # URI / wildcard names. Pulled in only via the legacy rustls 0.21 chain
     # from aws-smithy-http-client. The 0.103 line we actively use is patched.
@@ -84,23 +108,17 @@ ignore = [
     # https://rustsec.org/advisories/RUSTSEC-2026-0104
     { id = "RUSTSEC-2026-0104", reason = "only affects rustls-webpki 0.101 from legacy aws-smithy/rustls 0.21 chain" },
 
+    # rand 0.8.5: soundness issue only when ThreadRng reseeds inside a custom
+    # logger. Reached through several transitive chains. LanceDB does not use
+    # rand from a custom logger; upgrade once all pinned chains accept 0.8.6+.
+    # https://rustsec.org/advisories/RUSTSEC-2026-0097
+    { id = "RUSTSEC-2026-0097", reason = "transitive rand 0.8.5; LanceDB does not call ThreadRng from custom logging" },
+
     # pyo3 advisories in the Python bindings; tracked pending a patched pyo3 release.
     # https://rustsec.org/advisories/RUSTSEC-2026-0176
     # https://rustsec.org/advisories/RUSTSEC-2026-0177
     { id = "RUSTSEC-2026-0176", reason = "pyo3 in Python bindings; awaiting patched pyo3 release" },
     { id = "RUSTSEC-2026-0177", reason = "pyo3 in Python bindings; awaiting patched pyo3 release" },
-
-    # quick-xml < 0.41.0: quadratic runtime on duplicate attribute names (DoS).
-    # quick-xml < 0.41.0: unbounded namespace-declaration allocation in NsReader (DoS).
-    # Pulled in transitively by inferno (dev-only flame-graph dep), lance-namespace-impls
-    # (git dep from lance), and opendal/reqsign (cloud storage XML parsing). The XML
-    # parsed by opendal/reqsign comes from trusted cloud-storage endpoints (S3, GCS,
-    # Azure), not attacker-controlled input. Clearing requires upstream crates to migrate
-    # to quick-xml >= 0.41.0.
-    # https://rustsec.org/advisories/RUSTSEC-2026-0194
-    # https://rustsec.org/advisories/RUSTSEC-2026-0195
-    { id = "RUSTSEC-2026-0194", reason = "transitive via inferno/lance/opendal; XML from trusted cloud endpoints, not attacker-controlled" },
-    { id = "RUSTSEC-2026-0195", reason = "transitive via inferno/lance/opendal; XML from trusted cloud endpoints, not attacker-controlled" },
 ]
 
 # ---------------------------------------------------------------------------

docs/mkdocs.yml

@@ -51,11 +51,6 @@ plugins:
           paths: [../python/python]
           options:
             docstring_style: numpy
-            docstring_options:
-              # Attributes documented in a `Parameters` section, and pydantic
-              # dataclasses whose `__init__` griffe cannot see statically, both
-              # trip this check. It reports nothing actionable here.
-              warn_unknown_params: false
             heading_level: 3
             show_signature_annotations: true
             show_root_heading: true

docs/openapi.yml

@@ -453,16 +453,6 @@ paths:
                     The metric type to use for the index. l2, Cosine, Dot are supported.
                 index_type:
                   type: string
-                custom_stop_words:
-                  type: [array, "null"]
-                  items:
-                    type: string
-                  description: |
-                    The custom stop-word list for an FTS index. A non-null
-                    array replaces the language's built-in stop-word list and is only
-                    applied when remove_stop_words is enabled. Null uses the built-in
-                    language list, while an empty array explicitly replaces it with no
-                    stop words.
       responses:
         "200":
           description: Index successfully created
@@ -520,4 +510,4 @@ paths:
         "401":
           $ref: "#/components/responses/unauthorized"
         "404":
-          $ref: "#/components/responses/not_found"
+          $ref: "#/components/responses/not_found"

docs/src/java/java.md

@@ -14,7 +14,7 @@ Add the following dependency to your `pom.xml`:
 <dependency>
     <groupId>com.lancedb</groupId>
     <artifactId>lancedb-core</artifactId>
-    <version>0.37.1</version>
+    <version>0.31.0-beta.6</version>
 </dependency>
 ```
 
@@ -249,57 +249,6 @@ try (BufferAllocator allocator = new RootAllocator();
 }
 ```
 
-### Creating an Empty Table
-
-To create an empty table, send an Arrow IPC stream that contains the table schema and no record batches.
-The schema in the IPC stream becomes the table schema, and rows can be inserted later.
-
-```java
-import org.lance.namespace.model.CreateTableRequest;
-import org.lance.namespace.model.CreateTableResponse;
-import org.apache.arrow.memory.BufferAllocator;
-import org.apache.arrow.memory.RootAllocator;
-import org.apache.arrow.vector.VectorSchemaRoot;
-import org.apache.arrow.vector.ipc.ArrowStreamWriter;
-import org.apache.arrow.vector.types.FloatingPointPrecision;
-import org.apache.arrow.vector.types.pojo.ArrowType;
-import org.apache.arrow.vector.types.pojo.Field;
-import org.apache.arrow.vector.types.pojo.FieldType;
-import org.apache.arrow.vector.types.pojo.Schema;
-
-import java.io.ByteArrayOutputStream;
-import java.nio.channels.Channels;
-import java.util.Arrays;
-
-Schema schema = new Schema(Arrays.asList(
-    new Field("id", FieldType.nullable(new ArrowType.Int(32, true)), null),
-    new Field("name", FieldType.nullable(new ArrowType.Utf8()), null),
-    new Field("embedding",
-        FieldType.nullable(new ArrowType.FixedSizeList(128)),
-        Arrays.asList(new Field("item",
-            FieldType.nullable(new ArrowType.FloatingPoint(FloatingPointPrecision.SINGLE)),
-            null)))
-));
-
-byte[] emptyTableData;
-try (BufferAllocator allocator = new RootAllocator();
-     VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator)) {
-    root.setRowCount(0);
-
-    ByteArrayOutputStream out = new ByteArrayOutputStream();
-    try (ArrowStreamWriter writer = new ArrowStreamWriter(root, null, Channels.newChannel(out))) {
-        writer.start();
-        writer.end();
-    }
-    emptyTableData = out.toByteArray();
-}
-
-CreateTableRequest request = new CreateTableRequest();
-request.setId(Arrays.asList("my_namespace", "empty_table"));
-
-CreateTableResponse response = namespaceClient.createTable(request, emptyTableData);
-```
-
 ### Insert
 
 ```java
@@ -482,88 +431,9 @@ query.setVector(vector);
 byte[] result = namespaceClient.queryTable(query);
 ```
 
-## Indexing
+### Reading Query Results
 
-The Java SDK exposes the REST namespace index operations through the same `LanceNamespace` client.
-Index creation runs asynchronously, so use `listTableIndices` or `describeTableIndexStats` to check progress.
-
-### Creating a Vector Index
-
-```java
-import org.lance.namespace.model.CreateTableIndexRequest;
-import org.lance.namespace.model.CreateTableIndexResponse;
-
-CreateTableIndexRequest request = new CreateTableIndexRequest();
-request.setId(Arrays.asList("my_namespace", "my_table"));
-request.setColumn("embedding");
-request.setIndexType("IVF_PQ");
-request.setDistanceType("cosine");
-request.setName("embedding_idx");
-
-CreateTableIndexResponse response = namespaceClient.createTableIndex(request);
-System.out.println("Index transaction: " + response.getTransactionId());
-```
-
-### Creating a Scalar Index
-
-```java
-import org.lance.namespace.model.CreateTableIndexRequest;
-import org.lance.namespace.model.CreateTableScalarIndexResponse;
-
-CreateTableIndexRequest request = new CreateTableIndexRequest();
-request.setId(Arrays.asList("my_namespace", "my_table"));
-request.setColumn("category");
-request.setIndexType("BTREE");
-request.setName("category_idx");
-
-CreateTableScalarIndexResponse response = namespaceClient.createTableScalarIndex(request);
-System.out.println("Index transaction: " + response.getTransactionId());
-```
-
-### Creating a Full Text Search Index
-
-```java
-import org.lance.namespace.model.CreateTableIndexRequest;
-import org.lance.namespace.model.CreateTableScalarIndexResponse;
-
-CreateTableIndexRequest request = new CreateTableIndexRequest();
-request.setId(Arrays.asList("my_namespace", "my_table"));
-request.setColumn("text_column");
-request.setIndexType("FTS");
-request.setName("text_idx");
-request.setBaseTokenizer("simple");
-request.setLowerCase(true);
-request.setWithPosition(true);
-
-CreateTableScalarIndexResponse response = namespaceClient.createTableScalarIndex(request);
-System.out.println("Index transaction: " + response.getTransactionId());
-```
-
-### Listing Indexes
-
-```java
-import org.lance.namespace.model.IndexContent;
-import org.lance.namespace.model.ListTableIndicesRequest;
-import org.lance.namespace.model.ListTableIndicesResponse;
-
-ListTableIndicesRequest request = new ListTableIndicesRequest();
-request.setId(Arrays.asList("my_namespace", "my_table"));
-
-ListTableIndicesResponse response = namespaceClient.listTableIndices(request);
-for (IndexContent index : response.getIndexes()) {
-    System.out.println(index.getIndexName() + ": " + index.getStatus());
-}
-```
-
-!!! note
-    The current Java namespace API exposes index type, index name, distance type, and full text search tokenizer options.
-    IVF training parameters such as `num_partitions` are not exposed by `CreateTableIndexRequest` yet.
-    To make those configurable from Java, the namespace API must add those fields first.
-
-## Reading Query Results
-
-Query results are returned as bytes in Apache Arrow IPC file format. Put the byte-channel
-adapter behind a small helper so query code can work with `ArrowFileReader` directly:
+Query results are returned in Apache Arrow IPC file format. Here's how to read them:
 
 ```java
 import org.apache.arrow.vector.ipc.ArrowFileReader;
@@ -571,50 +441,45 @@ import org.apache.arrow.vector.VectorSchemaRoot;
 import org.apache.arrow.memory.BufferAllocator;
 import org.apache.arrow.memory.RootAllocator;
 
-import java.io.IOException;
 import java.nio.ByteBuffer;
 import java.nio.channels.SeekableByteChannel;
 
-final class ArrowIpc {
-    static ArrowFileReader openFileReader(byte[] data, BufferAllocator allocator) throws IOException {
-        return new ArrowFileReader(new ByteArraySeekableByteChannel(data), allocator);
+// Helper class to read Arrow data from byte array
+class ByteArraySeekableByteChannel implements SeekableByteChannel {
+    private final byte[] data;
+    private long position = 0;
+    private boolean isOpen = true;
+
+    public ByteArraySeekableByteChannel(byte[] data) {
+        this.data = data;
     }
 
-    private static final class ByteArraySeekableByteChannel implements SeekableByteChannel {
-        private final byte[] data;
-        private long position = 0;
-        private boolean isOpen = true;
-
-        private ByteArraySeekableByteChannel(byte[] data) {
-            this.data = data;
-        }
-
-        @Override
-        public int read(ByteBuffer dst) {
-            int remaining = dst.remaining();
-            int available = (int) (data.length - position);
-            if (available <= 0) return -1;
-            int toRead = Math.min(remaining, available);
-            dst.put(data, (int) position, toRead);
-            position += toRead;
-            return toRead;
-        }
-
-        @Override public long position() { return position; }
-        @Override public SeekableByteChannel position(long newPosition) { position = newPosition; return this; }
-        @Override public long size() { return data.length; }
-        @Override public boolean isOpen() { return isOpen; }
-        @Override public void close() { isOpen = false; }
-        @Override public int write(ByteBuffer src) { throw new UnsupportedOperationException(); }
-        @Override public SeekableByteChannel truncate(long size) { throw new UnsupportedOperationException(); }
+    @Override
+    public int read(ByteBuffer dst) {
+        int remaining = dst.remaining();
+        int available = (int) (data.length - position);
+        if (available <= 0) return -1;
+        int toRead = Math.min(remaining, available);
+        dst.put(data, (int) position, toRead);
+        position += toRead;
+        return toRead;
     }
+
+    @Override public long position() { return position; }
+    @Override public SeekableByteChannel position(long newPosition) { position = newPosition; return this; }
+    @Override public long size() { return data.length; }
+    @Override public boolean isOpen() { return isOpen; }
+    @Override public void close() { isOpen = false; }
+    @Override public int write(ByteBuffer src) { throw new UnsupportedOperationException(); }
+    @Override public SeekableByteChannel truncate(long size) { throw new UnsupportedOperationException(); }
 }
 
 // Read query results
 byte[] queryResult = namespaceClient.queryTable(query);
 
 try (BufferAllocator allocator = new RootAllocator();
-     ArrowFileReader reader = ArrowIpc.openFileReader(queryResult, allocator)) {
+     ArrowFileReader reader = new ArrowFileReader(
+         new ByteArraySeekableByteChannel(queryResult), allocator)) {
 
     for (int i = 0; i < reader.getRecordBlocks().size(); i++) {
         reader.loadRecordBatch(reader.getRecordBlocks().get(i));

docs/src/js/_media/CONTRIBUTING.md

@@ -1,7 +1,7 @@
 # Contributing to LanceDB Typescript
 
 This document outlines the process for contributing to LanceDB Typescript.
-For general contribution guidelines, see [CONTRIBUTING.md](https://github.com/lancedb/lancedb/blob/main/CONTRIBUTING.md).
+For general contribution guidelines, see [CONTRIBUTING.md](../CONTRIBUTING.md).
 
 ## Project layout
 

docs/src/js/classes/Branches.md

@@ -83,24 +83,6 @@ Delete a branch.
 
 ***
 
-### diff()
-
-```ts
-diff(fromBranch): Promise<BranchDiff>
-```
-
-Compare a branch against main without modifying either branch.
-
-#### Parameters
-
-* **fromBranch**: `string`
-
-#### Returns
-
-`Promise`&lt;[`BranchDiff`](../interfaces/BranchDiff.md)&gt;
-
-***
-
 ### list()
 
 ```ts
@@ -112,28 +94,3 @@ List all branches, mapping name to branch metadata.
 #### Returns
 
 `Promise`&lt;`Record`&lt;`string`, [`BranchContents`](BranchContents.md)&gt;&gt;
-
-***
-
-### merge()
-
-```ts
-merge(fromBranch, dryRun): Promise<MergeBranchResult>
-```
-
-Merge a branch into main.
-
-Set `dryRun` to `true` to preview the merge. A rejected merge resolves
-with `status: "rejected"` instead of throwing.
-
-#### Parameters
-
-* **fromBranch**: `string`
-    Branch to merge from.
-
-* **dryRun**: `boolean` = `false`
-    When true, only preview the merge. Defaults to false.
-
-#### Returns
-
-`Promise`&lt;[`MergeBranchResult`](../interfaces/MergeBranchResult.md)&gt;

docs/src/js/classes/Connection.md

@@ -25,27 +25,6 @@ the underlying connection has been closed.
 
 ## Methods
 
-### cancelJob()
-
-```ts
-abstract cancelJob(jobId): Promise<boolean>
-```
-
-Request cancellation of a server-side job by id.
-
-Resolves to true if the server accepted the cancellation, false if no
-such job exists. Cancelling an already-terminal job is a no-op success.
-
-#### Parameters
-
-* **jobId**: `string`
-
-#### Returns
-
-`Promise`&lt;`boolean`&gt;
-
-***
-
 ### cloneTable()
 
 ```ts
@@ -386,26 +365,6 @@ Drop an existing table.
 
 ***
 
-### getJob()
-
-```ts
-abstract getJob(jobId): Promise<null | JobDescription>
-```
-
-Describe a single server-side job by id.
-
-Resolves to `null` when the server has no such job.
-
-#### Parameters
-
-* **jobId**: `string`
-
-#### Returns
-
-`Promise`&lt;`null` \| [`JobDescription`](../interfaces/JobDescription.md)&gt;
-
-***
-
 ### isOpen()
 
 ```ts
@@ -420,62 +379,6 @@ Return true if the connection has not been closed
 
 ***
 
-### job()
-
-```ts
-abstract job(jobId): Job
-```
-
-A [Job](Job.md) handle for a server-side job by id.
-
-The handle is constructed without a server round trip; an unknown id
-surfaces when the handle is used. Dropping the handle has no effect on
-the job itself.
-
-#### Parameters
-
-* **jobId**: `string`
-
-#### Returns
-
-[`Job`](Job.md)
-
-***
-
-### jobHistory()
-
-```ts
-abstract jobHistory(jobId?): Promise<Table<any>>
-```
-
-The lifecycle event history of a server-side job, as an Arrow table.
-
-Lists history across all jobs when `jobId` is omitted.
-
-#### Parameters
-
-* **jobId?**: `string`
-
-#### Returns
-
-`Promise`&lt;`Table`&lt;`any`&gt;&gt;
-
-***
-
-### listJobs()
-
-```ts
-abstract listJobs(): Promise<JobInfo[]>
-```
-
-List server-side jobs across the database's tables.
-
-#### Returns
-
-`Promise`&lt;[`JobInfo`](../interfaces/JobInfo.md)[]&gt;
-
-***
-
 ### listNamespaces()
 
 ```ts

docs/src/js/classes/Job.md

@@ -1,83 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / Job
-
-# Class: Job
-
-A handle to an operation that may still be running.
-
-## Constructors
-
-### new Job()
-
-```ts
-new Job(): Job
-```
-
-#### Returns
-
-[`Job`](Job.md)
-
-## Accessors
-
-### id
-
-```ts
-get id(): null | string
-```
-
-Identifies the operation on the server that is running it. Operations
-that run in this process have no server id. The value is opaque.
-
-#### Returns
-
-`null` \| `string`
-
-## Methods
-
-### cancel()
-
-```ts
-cancel(): Promise<void>
-```
-
-Request cancellation. Cancelling a finished operation is a no-op.
-
-#### Returns
-
-`Promise`&lt;`void`&gt;
-
-***
-
-### status()
-
-```ts
-status(): Promise<string>
-```
-
-The operation's current lifecycle state: "running", "finished",
-"failed", or "cancelled".
-
-A point snapshot; unlike [Job.wait](Job.md#wait) it does not block or reject
-on a terminal failure state. States a newer server reports that this
-client version does not know pass through as-is.
-
-#### Returns
-
-`Promise`&lt;`string`&gt;
-
-***
-
-### wait()
-
-```ts
-wait(): Promise<void>
-```
-
-Wait until the operation reaches a terminal state.
-
-#### Returns
-
-`Promise`&lt;`void`&gt;

docs/src/js/classes/MergeInsertBuilder.md

@@ -76,23 +76,24 @@ the query optimizer chooses a suboptimal path.
 
 ***
 
-### useLsm()
+### useLsmWrite()
 
 ```ts
-useLsm(enable): MergeInsertBuilder
+useLsmWrite(useLsmWrite): MergeInsertBuilder
 ```
 
-Control MemWAL routing for this merge.
+Controls whether the merge uses the MemWAL LSM write path.
 
 By default (unset), a `mergeInsert` on a table with an LSM write spec is
-routed through Lance's MemWAL shard writer, and a table without one uses the
-standard path.
+routed through Lance's MemWAL shard writer, and a table without one uses
+the standard path. Pass `false` to force the standard path even when a
+spec is set. Pass `true` to require a spec — `mergeInsert` rejects if none
+is installed.
 
 #### Parameters
 
-* **enable**: `boolean`
-    `true` forces MemWAL routing and errors if the table has no
-    LSM write spec. `false` forces the standard write path even when a spec is set.
+* **useLsmWrite**: `boolean`
+    Whether to use the LSM write path.
 
 #### Returns
 

docs/src/js/classes/Query.md

@@ -33,7 +33,7 @@ protected inner: Query | Promise<Query>;
 ### analyzePlan()
 
 ```ts
-analyzePlan(distributedMetrics?): Promise<string>
+analyzePlan(): Promise<string>
 ```
 
 Executes the query and returns the physical query plan annotated with runtime metrics.
@@ -41,12 +41,6 @@ Executes the query and returns the physical query plan annotated with runtime me
 This is useful for debugging and performance analysis, as it shows how the query was executed
 and includes metrics such as elapsed time, rows processed, and I/O statistics.
 
-#### Parameters
-
-* **distributedMetrics?**: [`AnalyzePlanDistributedMetrics`](../type-aliases/AnalyzePlanDistributedMetrics.md)
-    How distributed worker metrics are displayed for remote query plans.
-    Defaults to `"aggregate"`.
-
 #### Returns
 
 `Promise`&lt;`string`&gt;
@@ -497,42 +491,6 @@ ArrowTable.
 
 ***
 
-### useLsm()
-
-```ts
-useLsm(enable): this
-```
-
-Control MemWAL read routing for this query.
-
-By default (unset), when the table carries a MemWAL write spec (see
-[Table#setLsmWriteSpec](Table.md#setlsmwritespec)), reads are routed through the LSM scanner so
-they also return data written via the `mergeInsert` LSM path that has not yet
-been compacted into the base table (the active/frozen in-memory memtables and
-the flushed generations), deduplicated by primary key; a table without a spec
-reads the base table.
-
-#### Parameters
-
-* **enable**: `boolean`
-    `true` forces the LSM scanner and errors if the table has no
-    MemWAL write spec. `false` bypasses the MemWAL and reads the base table only,
-    even when a spec is present.
-    Note: the LSM scanner does not support every query shape (e.g. reranking,
-    hybrid search, `orderBy`). On a MemWAL table those shapes error unless
-    `useLsm(false)` is set, because a base-only read would silently exclude
-    un-compacted MemWAL data.
-
-#### Returns
-
-`this`
-
-#### Inherited from
-
-`StandardQueryBase.useLsm`
-
-***
-
 ### where()
 
 ```ts

docs/src/js/classes/QueryBase.md

@@ -38,7 +38,7 @@ protected inner: NativeQueryType | Promise<NativeQueryType>;
 ### analyzePlan()
 
 ```ts
-analyzePlan(distributedMetrics?): Promise<string>
+analyzePlan(): Promise<string>
 ```
 
 Executes the query and returns the physical query plan annotated with runtime metrics.
@@ -46,12 +46,6 @@ Executes the query and returns the physical query plan annotated with runtime me
 This is useful for debugging and performance analysis, as it shows how the query was executed
 and includes metrics such as elapsed time, rows processed, and I/O statistics.
 
-#### Parameters
-
-* **distributedMetrics?**: [`AnalyzePlanDistributedMetrics`](../type-aliases/AnalyzePlanDistributedMetrics.md)
-    How distributed worker metrics are displayed for remote query plans.
-    Defaults to `"aggregate"`.
-
 #### Returns
 
 `Promise`&lt;`string`&gt;

docs/src/js/classes/Table.md

@@ -295,29 +295,6 @@ await table.createIndex("my_float_col");
 
 ***
 
-### createIndexAsync()
-
-```ts
-abstract createIndexAsync(column, options?): Promise<Job>
-```
-
-Create an index, returning a handle to the indexing job.
-
-The job may already be complete when returned; callers must not assume
-the index exists until [Job.wait](Job.md#wait) resolves.
-
-#### Parameters
-
-* **column**: `string`
-
-* **options?**: `Partial`&lt;[`IndexOptions`](../interfaces/IndexOptions.md)&gt;
-
-#### Returns
-
-`Promise`&lt;[`Job`](Job.md)&gt;
-
-***
-
 ### currentBranch()
 
 ```ts
@@ -421,26 +398,6 @@ Drop an index from the table.
 
 ***
 
-### getLsmWriteSpec()
-
-```ts
-abstract getLsmWriteSpec(): Promise<undefined | LsmWriteSpec>
-```
-
-Read the [LsmWriteSpec](../interfaces/LsmWriteSpec.md) currently installed on this table.
-
-Resolves to `undefined` when the MemWAL LSM write path is not enabled (no
-spec has been set, or it was removed with [Table#unsetLsmWriteSpec](Table.md#unsetlsmwritespec)).
-The returned spec — including its `maintainedIndexes` and
-`writerConfigDefaults` — mirrors what was passed to
-[Table#setLsmWriteSpec](Table.md#setlsmwritespec).
-
-#### Returns
-
-`Promise`&lt;`undefined` \| [`LsmWriteSpec`](../interfaces/LsmWriteSpec.md)&gt;
-
-***
-
 ### indexStats()
 
 ```ts
@@ -957,32 +914,6 @@ Return the table as an arrow table
 
 ***
 
-### tokenize()
-
-```ts
-abstract tokenize(query, options): Promise<FtsToken[]>
-```
-
-Tokenize a full-text search query using the tokenizer configured on an FTS index.
-
-Specify exactly one of `column` or `indexName`.
-
-Model-backed tokenizers such as `jieba/*` and `lindera/*` are rebuilt in
-the client process from index metadata. For remote tables, this means the
-same tokenizer model files must also exist locally.
-
-#### Parameters
-
-* **query**: `string`
-
-* **options**: [`TokenizeTableOptions`](../type-aliases/TokenizeTableOptions.md)
-
-#### Returns
-
-`Promise`&lt;[`FtsToken`](../interfaces/FtsToken.md)[]&gt;
-
-***
-
 ### unsetLsmWriteSpec()
 
 ```ts

docs/src/js/classes/TakeQuery.md

@@ -29,7 +29,7 @@ protected inner: TakeQuery | Promise<TakeQuery>;
 ### analyzePlan()
 
 ```ts
-analyzePlan(distributedMetrics?): Promise<string>
+analyzePlan(): Promise<string>
 ```
 
 Executes the query and returns the physical query plan annotated with runtime metrics.
@@ -37,12 +37,6 @@ Executes the query and returns the physical query plan annotated with runtime me
 This is useful for debugging and performance analysis, as it shows how the query was executed
 and includes metrics such as elapsed time, rows processed, and I/O statistics.
 
-#### Parameters
-
-* **distributedMetrics?**: [`AnalyzePlanDistributedMetrics`](../type-aliases/AnalyzePlanDistributedMetrics.md)
-    How distributed worker metrics are displayed for remote query plans.
-    Defaults to `"aggregate"`.
-
 #### Returns
 
 `Promise`&lt;`string`&gt;
@@ -273,29 +267,6 @@ ArrowTable.
 
 ***
 
-### useLsm()
-
-```ts
-useLsm(enable): this
-```
-
-Control MemWAL read routing for this take query.
-
-`false` bypasses the MemWAL and reads the base table only — the escape hatch,
-since take-by-row-id/offset is not supported on the LSM scanner and, on a
-MemWAL table, auto-routes to it and errors otherwise.
-
-#### Parameters
-
-* **enable**: `boolean`
-    `false` reads the base table only.
-
-#### Returns
-
-`this`
-
-***
-
 ### withRowId()
 
 ```ts

docs/src/js/classes/VectorQuery.md

@@ -51,7 +51,7 @@ addQueryVector(vector): VectorQuery
 ### analyzePlan()
 
 ```ts
-analyzePlan(distributedMetrics?): Promise<string>
+analyzePlan(): Promise<string>
 ```
 
 Executes the query and returns the physical query plan annotated with runtime metrics.
@@ -59,12 +59,6 @@ Executes the query and returns the physical query plan annotated with runtime me
 This is useful for debugging and performance analysis, as it shows how the query was executed
 and includes metrics such as elapsed time, rows processed, and I/O statistics.
 
-#### Parameters
-
-* **distributedMetrics?**: [`AnalyzePlanDistributedMetrics`](../type-aliases/AnalyzePlanDistributedMetrics.md)
-    How distributed worker metrics are displayed for remote query plans.
-    Defaults to `"aggregate"`.
-
 #### Returns
 
 `Promise`&lt;`string`&gt;
@@ -746,42 +740,6 @@ ArrowTable.
 
 ***
 
-### useLsm()
-
-```ts
-useLsm(enable): this
-```
-
-Control MemWAL read routing for this query.
-
-By default (unset), when the table carries a MemWAL write spec (see
-[Table#setLsmWriteSpec](Table.md#setlsmwritespec)), reads are routed through the LSM scanner so
-they also return data written via the `mergeInsert` LSM path that has not yet
-been compacted into the base table (the active/frozen in-memory memtables and
-the flushed generations), deduplicated by primary key; a table without a spec
-reads the base table.
-
-#### Parameters
-
-* **enable**: `boolean`
-    `true` forces the LSM scanner and errors if the table has no
-    MemWAL write spec. `false` bypasses the MemWAL and reads the base table only,
-    even when a spec is present.
-    Note: the LSM scanner does not support every query shape (e.g. reranking,
-    hybrid search, `orderBy`). On a MemWAL table those shapes error unless
-    `useLsm(false)` is set, because a base-only read would silently exclude
-    un-compacted MemWAL data.
-
-#### Returns
-
-`this`
-
-#### Inherited from
-
-`StandardQueryBase.useLsm`
-
-***
-
 ### where()
 
 ```ts

docs/src/js/functions/instrumentLanceDbMetrics.md

@@ -1,42 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / instrumentLanceDbMetrics
-
-# Function: instrumentLanceDbMetrics()
-
-```ts
-function instrumentLanceDbMetrics(meterProvider?): boolean
-```
-
-Register LanceDB metrics as OpenTelemetry observable instruments.
-
-Installs a process-global metrics recorder and creates one observable
-instrument per LanceDB metric (currently object store request counts, bytes,
-latency, errors, and throttles) on the given (or global) `MeterProvider`. The
-configured `MetricReader` then collects them on its own schedule.
-
-Counters and gauges map directly to observable counters/gauges. Because
-OpenTelemetry has no asynchronous histogram instrument, each histogram is
-exported Prometheus-style as cumulative `le` bucket counts (`<name>_bucket`,
-with an `le` attribute) plus `<name>_count` and `<name>_sum`.
-
-Requires `@opentelemetry/api` (a dependency) and, to actually export, an
-OpenTelemetry SDK such as `@opentelemetry/sdk-metrics`.
-
-## Parameters
-
-* **meterProvider?**: `MeterProvider`
-    The provider to register instruments on. Defaults to the
-    global provider from `@opentelemetry/api`.
-
-## Returns
-
-`boolean`
-
-`true` if the recorder is installed and instruments are registered.
-  `false` if a different `metrics` recorder is already installed in this
-  process (only one global recorder is permitted), in which case a warning is
-  emitted and no instruments are created. Calling this more than once is safe;
-  instruments are created only on the first successful call.

docs/src/js/functions/tokenize.md

@@ -1,26 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / tokenize
-
-# Function: tokenize()
-
-```ts
-function tokenize(query, options?): Promise<FtsToken[]>
-```
-
-Tokenize a full-text search query using an explicit tokenizer.
-
-This does not require a table or FTS index. The tokenizer options match
-[Index.fts](../classes/Index.md#fts).
-
-## Parameters
-
-* **query**: `string`
-
-* **options?**: `Partial`&lt;[`TokenizeOptions`](../interfaces/TokenizeOptions.md)&gt;
-
-## Returns
-
-`Promise`&lt;[`FtsToken`](../interfaces/FtsToken.md)[]&gt;

docs/src/js/globals.md

@@ -25,7 +25,6 @@
 - [Connection](classes/Connection.md)
 - [HeaderProvider](classes/HeaderProvider.md)
 - [Index](classes/Index.md)
-- [Job](classes/Job.md)
 - [MakeArrowTableOptions](classes/MakeArrowTableOptions.md)
 - [MatchQuery](classes/MatchQuery.md)
 - [MergeInsertBuilder](classes/MergeInsertBuilder.md)
@@ -53,11 +52,6 @@
 - [AddDataOptions](interfaces/AddDataOptions.md)
 - [AddResult](interfaces/AddResult.md)
 - [AlterColumnsResult](interfaces/AlterColumnsResult.md)
-- [BranchColumnChange](interfaces/BranchColumnChange.md)
-- [BranchColumnSummary](interfaces/BranchColumnSummary.md)
-- [BranchDiff](interfaces/BranchDiff.md)
-- [BranchIndexSummary](interfaces/BranchIndexSummary.md)
-- [BranchRowCountSummary](interfaces/BranchRowCountSummary.md)
 - [ClientConfig](interfaces/ClientConfig.md)
 - [ColumnAlteration](interfaces/ColumnAlteration.md)
 - [ColumnOrdering](interfaces/ColumnOrdering.md)
@@ -78,7 +72,6 @@
 - [FragmentStatistics](interfaces/FragmentStatistics.md)
 - [FragmentSummaryStats](interfaces/FragmentSummaryStats.md)
 - [FtsOptions](interfaces/FtsOptions.md)
-- [FtsToken](interfaces/FtsToken.md)
 - [FullTextQuery](interfaces/FullTextQuery.md)
 - [FullTextSearchOptions](interfaces/FullTextSearchOptions.md)
 - [HnswPqOptions](interfaces/HnswPqOptions.md)
@@ -89,15 +82,9 @@
 - [IvfFlatOptions](interfaces/IvfFlatOptions.md)
 - [IvfPqOptions](interfaces/IvfPqOptions.md)
 - [IvfRqOptions](interfaces/IvfRqOptions.md)
-- [JobDescription](interfaces/JobDescription.md)
-- [JobFailureInfo](interfaces/JobFailureInfo.md)
-- [JobInfo](interfaces/JobInfo.md)
 - [ListNamespacesOptions](interfaces/ListNamespacesOptions.md)
 - [ListNamespacesResponse](interfaces/ListNamespacesResponse.md)
 - [LsmWriteSpec](interfaces/LsmWriteSpec.md)
-- [MergeBlocker](interfaces/MergeBlocker.md)
-- [MergeBranchResult](interfaces/MergeBranchResult.md)
-- [MergePreview](interfaces/MergePreview.md)
 - [MergeResult](interfaces/MergeResult.md)
 - [NativeOAuthConfig](interfaces/NativeOAuthConfig.md)
 - [OAuthConfig](interfaces/OAuthConfig.md)
@@ -120,7 +107,6 @@
 - [TimeoutConfig](interfaces/TimeoutConfig.md)
 - [TlsConfig](interfaces/TlsConfig.md)
 - [TokenResponse](interfaces/TokenResponse.md)
-- [TokenizeOptions](interfaces/TokenizeOptions.md)
 - [UpdateFieldMetadataResult](interfaces/UpdateFieldMetadataResult.md)
 - [UpdateOptions](interfaces/UpdateOptions.md)
 - [UpdateResult](interfaces/UpdateResult.md)
@@ -130,8 +116,6 @@
 
 ## Type Aliases
 
-- [AnalyzePlanDistributedMetrics](type-aliases/AnalyzePlanDistributedMetrics.md)
-- [BaseTokenizer](type-aliases/BaseTokenizer.md)
 - [Data](type-aliases/Data.md)
 - [DataLike](type-aliases/DataLike.md)
 - [FieldLike](type-aliases/FieldLike.md)
@@ -141,15 +125,12 @@
 - [RecordBatchLike](type-aliases/RecordBatchLike.md)
 - [SchemaLike](type-aliases/SchemaLike.md)
 - [TableLike](type-aliases/TableLike.md)
-- [TokenizeTableOptions](type-aliases/TokenizeTableOptions.md)
 
 ## Functions
 
 - [RecordBatchIterator](functions/RecordBatchIterator.md)
 - [connect](functions/connect.md)
 - [connectNamespace](functions/connectNamespace.md)
-- [instrumentLanceDbMetrics](functions/instrumentLanceDbMetrics.md)
 - [makeArrowTable](functions/makeArrowTable.md)
 - [packBits](functions/packBits.md)
 - [permutationBuilder](functions/permutationBuilder.md)
-- [tokenize](functions/tokenize.md)

docs/src/js/interfaces/BranchColumnChange.md

@@ -1,33 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / BranchColumnChange
-
-# Interface: BranchColumnChange
-
-A column whose definition differs between main and the branch.
-
-## Properties
-
-### branch
-
-```ts
-branch: BranchColumnSummary;
-```
-
-***
-
-### main
-
-```ts
-main: BranchColumnSummary;
-```
-
-***
-
-### name
-
-```ts
-name: string;
-```

docs/src/js/interfaces/BranchColumnSummary.md

@@ -1,33 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / BranchColumnSummary
-
-# Interface: BranchColumnSummary
-
-Summary of a column in a branch diff.
-
-## Properties
-
-### dataType
-
-```ts
-dataType: string;
-```
-
-***
-
-### name
-
-```ts
-name: string;
-```
-
-***
-
-### nullable
-
-```ts
-nullable: boolean;
-```

docs/src/js/interfaces/BranchDiff.md

@@ -1,129 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / BranchDiff
-
-# Interface: BranchDiff
-
-Read-only comparison of a branch against main.
-
-## Properties
-
-### addedColumns
-
-```ts
-addedColumns: BranchColumnSummary[];
-```
-
-***
-
-### addedIndexes
-
-```ts
-addedIndexes: BranchIndexSummary[];
-```
-
-***
-
-### baseMoved
-
-```ts
-baseMoved: boolean;
-```
-
-***
-
-### branchVersion
-
-```ts
-branchVersion: number;
-```
-
-***
-
-### changedColumns
-
-```ts
-changedColumns: BranchColumnChange[];
-```
-
-***
-
-### fromBranch
-
-```ts
-fromBranch: string;
-```
-
-***
-
-### mainVersion
-
-```ts
-mainVersion: number;
-```
-
-***
-
-### mergeBlockers
-
-```ts
-mergeBlockers: MergeBlocker[];
-```
-
-***
-
-### mergeable
-
-```ts
-mergeable: boolean;
-```
-
-***
-
-### parentVersion
-
-```ts
-parentVersion: number;
-```
-
-***
-
-### removedColumns
-
-```ts
-removedColumns: BranchColumnSummary[];
-```
-
-***
-
-### removedIndexes
-
-```ts
-removedIndexes: BranchIndexSummary[];
-```
-
-***
-
-### rowCountBranch
-
-```ts
-rowCountBranch: number;
-```
-
-***
-
-### rowCountMain
-
-```ts
-rowCountMain: number;
-```
-
-***
-
-### rowSummary
-
-```ts
-rowSummary: BranchRowCountSummary;
-```

docs/src/js/interfaces/BranchIndexSummary.md

@@ -1,41 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / BranchIndexSummary
-
-# Interface: BranchIndexSummary
-
-Summary of an index in a branch diff.
-
-## Properties
-
-### columns
-
-```ts
-columns: string[];
-```
-
-***
-
-### indexName
-
-```ts
-indexName: string;
-```
-
-***
-
-### indexType?
-
-```ts
-optional indexType: string;
-```
-
-***
-
-### status
-
-```ts
-status: string;
-```

docs/src/js/interfaces/BranchRowCountSummary.md

@@ -1,57 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / BranchRowCountSummary
-
-# Interface: BranchRowCountSummary
-
-Row-level comparison between main and the branch.
-
-## Properties
-
-### deltaAvailable
-
-```ts
-deltaAvailable: boolean;
-```
-
-***
-
-### inputsChanged
-
-```ts
-inputsChanged: number;
-```
-
-***
-
-### newOnBase
-
-```ts
-newOnBase: number;
-```
-
-***
-
-### newOnBranch
-
-```ts
-newOnBranch: number;
-```
-
-***
-
-### staleRecompute
-
-```ts
-staleRecompute: number;
-```
-
-***
-
-### unchanged
-
-```ts
-unchanged: number;
-```

docs/src/js/interfaces/FtsOptions.md

@@ -23,7 +23,7 @@ whether to remove punctuation
 ### baseTokenizer?
 
 ```ts
-optional baseTokenizer: BaseTokenizer;
+optional baseTokenizer: "raw" | "simple" | "whitespace" | "ngram";
 ```
 
 The tokenizer to use when building the index.
@@ -37,38 +37,6 @@ The following tokenizers are available:
 
 "raw" - Raw tokenizer. This tokenizer does not split the text into tokens and indexes the entire text as a single token.
 
-"icu" - ICU dictionary-based word segmentation.
-
-"icu/split" - ICU segmentation with simple-style delimiter splitting.
-
-***
-
-### blockSize?
-
-```ts
-optional blockSize: 128 | 256;
-```
-
-Number of documents per compressed posting block.
-
-The default is 128. Supported values are 128 and 256. A value of 256 uses
-the experimental FTS V3 format and may introduce breaking changes.
-
-***
-
-### customStopWords?
-
-```ts
-optional customStopWords: string[];
-```
-
-Custom stop words that replace the built-in list for `language`.
-
-This option only affects tokenization when `removeStopWords` is true.
-
-`undefined` keeps the built-in language list. An empty array explicitly
-replaces it with no stop words.
-
 ***
 
 ### language?

docs/src/js/interfaces/FtsToken.md

@@ -1,29 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / FtsToken
-
-# Interface: FtsToken
-
-Token produced by the tokenizer configured on a full-text search index.
-
-## Properties
-
-### position
-
-```ts
-position: number;
-```
-
-Token position used by full-text query matching.
-
-***
-
-### text
-
-```ts
-text: string;
-```
-
-Token text after tokenizer filters have been applied.

docs/src/js/interfaces/JobDescription.md

@@ -1,66 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / JobDescription
-
-# Interface: JobDescription
-
-A described job from `Connection.getJob`.
-
-## Properties
-
-### creationMs
-
-```ts
-creationMs: number;
-```
-
-When the job was created, in milliseconds since the epoch.
-
-***
-
-### failure?
-
-```ts
-optional failure: JobFailureInfo;
-```
-
-Why the job failed, when the job is failed and the server reports a
-reason.
-
-***
-
-### jobId
-
-```ts
-jobId: string;
-```
-
-***
-
-### jobType
-
-```ts
-jobType: string;
-```
-
-***
-
-### specJson?
-
-```ts
-optional specJson: string;
-```
-
-The job-type-specific specification as a JSON string, when present.
-
-***
-
-### state
-
-```ts
-state: string;
-```
-
-Lifecycle state: "running", "finished", "failed", or "cancelled".

docs/src/js/interfaces/JobFailureInfo.md

@@ -1,33 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / JobFailureInfo
-
-# Interface: JobFailureInfo
-
-The server's account of why a job failed.
-
-## Properties
-
-### message?
-
-```ts
-optional message: string;
-```
-
-***
-
-### phase?
-
-```ts
-optional phase: string;
-```
-
-***
-
-### retryable?
-
-```ts
-optional retryable: boolean;
-```

docs/src/js/interfaces/JobInfo.md

@@ -1,58 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / JobInfo
-
-# Interface: JobInfo
-
-A row from `Connection.listJobs`: one server-side job.
-
-## Properties
-
-### createdAtMillis
-
-```ts
-createdAtMillis: number;
-```
-
-When the job was created, in milliseconds since the epoch.
-
-***
-
-### jobId
-
-```ts
-jobId: string;
-```
-
-The job id -- what `Connection.getJob` and `Connection.cancelJob`
-accept.
-
-***
-
-### jobType
-
-```ts
-jobType: string;
-```
-
-***
-
-### state
-
-```ts
-state: string;
-```
-
-Lifecycle state: "running", "finished", "failed", or "cancelled".
-
-***
-
-### table
-
-```ts
-table: string;
-```
-
-The table the job runs against, without URI or namespace.

docs/src/js/interfaces/MergeBlocker.md

@@ -1,25 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / MergeBlocker
-
-# Interface: MergeBlocker
-
-A reason why a branch cannot currently be merged.
-
-## Properties
-
-### code
-
-```ts
-code: string;
-```
-
-***
-
-### message
-
-```ts
-message: string;
-```

docs/src/js/interfaces/MergeBranchResult.md

@@ -1,46 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / MergeBranchResult
-
-# Interface: MergeBranchResult
-
-Result of previewing or attempting a branch merge.
-
-## Properties
-
-### diff
-
-```ts
-diff: BranchDiff;
-```
-
-***
-
-### mainVersionAfter?
-
-```ts
-optional mainVersionAfter: number;
-```
-
-***
-
-### preview
-
-```ts
-preview: MergePreview;
-```
-
-***
-
-### status
-
-```ts
-status:
-  | "unknown"
-  | "rejected"
-  | "ready"
-  | "notImplemented"
-  | "merged";
-```

docs/src/js/interfaces/MergePreview.md

@@ -1,17 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / MergePreview
-
-# Interface: MergePreview
-
-Changes that would be, or were, promoted by a branch merge.
-
-## Properties
-
-### promotedColumns
-
-```ts
-promotedColumns: string[];
-```

docs/src/js/interfaces/SplitRandomOptions.md

@@ -8,14 +8,6 @@
 
 ## Properties
 
-### clumpSize?
-
-```ts
-optional clumpSize: number;
-```
-
-***
-
 ### counts?
 
 ```ts

docs/src/js/interfaces/TableStatistics.md

@@ -44,7 +44,4 @@ The number of rows in the table
 totalBytes: number;
 ```
 
-The total size, in bytes, of the table's data files, index files, and
-overlay files
-
-Read from the manifest, so this excludes deletion files and manifests.
+The total number of bytes in the table

docs/src/js/interfaces/TokenizeOptions.md

@@ -1,124 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / TokenizeOptions
-
-# Interface: TokenizeOptions
-
-Options for tokenizing a full-text search query without a table index.
-
-## Properties
-
-### asciiFolding?
-
-```ts
-optional asciiFolding: boolean;
-```
-
-Whether to fold ASCII characters.
-
-***
-
-### baseTokenizer?
-
-```ts
-optional baseTokenizer: BaseTokenizer;
-```
-
-The tokenizer to use. The default is "simple".
-
-***
-
-### customStopWords?
-
-```ts
-optional customStopWords: string[];
-```
-
-Custom stop words that replace the built-in list for `language`.
-
-This option only affects tokenization when `removeStopWords` is true.
-
-`undefined` keeps the built-in language list. An empty array explicitly
-replaces it with no stop words.
-
-***
-
-### language?
-
-```ts
-optional language: string;
-```
-
-Language for stemming and stop words.
-
-***
-
-### lowercase?
-
-```ts
-optional lowercase: boolean;
-```
-
-Whether to lowercase tokens.
-
-***
-
-### maxTokenLength?
-
-```ts
-optional maxTokenLength: number;
-```
-
-Maximum token length; tokens longer than this are ignored.
-
-***
-
-### ngramMaxLength?
-
-```ts
-optional ngramMaxLength: number;
-```
-
-N-gram maximum length.
-
-***
-
-### ngramMinLength?
-
-```ts
-optional ngramMinLength: number;
-```
-
-N-gram minimum length.
-
-***
-
-### prefixOnly?
-
-```ts
-optional prefixOnly: boolean;
-```
-
-Whether to only emit token prefixes for the n-gram tokenizer.
-
-***
-
-### removeStopWords?
-
-```ts
-optional removeStopWords: boolean;
-```
-
-Whether to remove stop words.
-
-***
-
-### stem?
-
-```ts
-optional stem: boolean;
-```
-
-Whether to stem tokens.

docs/src/js/type-aliases/AnalyzePlanDistributedMetrics.md

@@ -1,11 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / AnalyzePlanDistributedMetrics
-
-# Type Alias: AnalyzePlanDistributedMetrics
-
-```ts
-type AnalyzePlanDistributedMetrics: "aggregate" | "per_worker" | "full";
-```

docs/src/js/type-aliases/BaseTokenizer.md

@@ -1,19 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / BaseTokenizer
-
-# Type Alias: BaseTokenizer
-
-```ts
-type BaseTokenizer:
-  | "simple"
-  | "whitespace"
-  | "raw"
-  | "ngram"
-  | "icu"
-  | "icu/split"
-  | `jieba/${string}`
-  | `lindera/${string}`;
-```

docs/src/js/type-aliases/TokenizeTableOptions.md

@@ -1,11 +0,0 @@
-[**@lancedb/lancedb**](../README.md) • **Docs**
-
-***
-
-[@lancedb/lancedb](../globals.md) / TokenizeTableOptions
-
-# Type Alias: TokenizeTableOptions
-
-```ts
-type TokenizeTableOptions: object | object;
-```

docs/src/python/python.md

@@ -26,18 +26,6 @@ is also an [asynchronous API client](#connections-asynchronous).
 
 ::: lancedb.db.DBConnection
 
-::: lancedb.Session
-
-## Namespaces (Synchronous)
-
-A namespace-backed connection resolves tables through a
-[Lance namespace](https://lance-format.github.io/lance-namespace/) service instead of
-listing a storage directory.
-
-::: lancedb.connect_namespace
-
-::: lancedb.namespace.LanceNamespaceDBConnection
-
 ## Tables (Synchronous)
 
 ::: lancedb.table.Table
@@ -46,12 +34,8 @@ listing a storage directory.
 
 ::: lancedb.table.FragmentSummaryStats
 
-::: lancedb.table.TableStatistics
-
 ::: lancedb.table.Tags
 
-::: lancedb.table.Branches
-
 ## Expressions
 
 Type-safe expression builder for filters and projections. Use these instead
@@ -78,46 +62,29 @@ of raw SQL strings with [where][lancedb.query.LanceQueryBuilder.where] and
 
 ::: lancedb.query.LanceHybridQueryBuilder
 
-::: lancedb.query.LanceEmptyQueryBuilder
-
-::: lancedb.query.LanceTakeQueryBuilder
-
-## Full text queries
-
-Structured full text queries can be passed to
-[Table.search][lancedb.table.Table.search] or
-[AsyncTable.search][lancedb.table.AsyncTable.search] in place of a query string,
-and combined with [BooleanQuery][lancedb.query.BooleanQuery].
-
-::: lancedb.query.FullTextQuery
-
-::: lancedb.query.MatchQuery
-
-::: lancedb.query.PhraseQuery
-
-::: lancedb.query.BoostQuery
-
-::: lancedb.query.MultiMatchQuery
-
-::: lancedb.query.BooleanQuery
-
-::: lancedb.query.FullTextOperator
-
-::: lancedb.query.Occur
-
 ## Embeddings
 
-::: lancedb.embeddings
-    options:
-      show_root_heading: false
-      show_root_toc_entry: false
+::: lancedb.embeddings.registry.EmbeddingFunctionRegistry
+
+::: lancedb.embeddings.base.EmbeddingFunctionConfig
+
+::: lancedb.embeddings.base.EmbeddingFunction
+
+::: lancedb.embeddings.base.TextEmbeddingFunction
+
+::: lancedb.embeddings.sentence_transformers.SentenceTransformerEmbeddings
+
+::: lancedb.embeddings.openai.OpenAIEmbeddings
+
+::: lancedb.embeddings.open_clip.OpenClipEmbeddings
 
 ## Remote configuration
 
-::: lancedb.remote
-    options:
-      show_root_heading: false
-      show_root_toc_entry: false
+::: lancedb.remote.ClientConfig
+
+::: lancedb.remote.TimeoutConfig
+
+::: lancedb.remote.RetryConfig
 
 ## Context
 
@@ -127,50 +94,11 @@ and combined with [BooleanQuery][lancedb.query.BooleanQuery].
 
 ## Full text search
 
-Pass `custom_stop_words` to [lancedb.index.FTS][]:
+Use [lancedb.table.Table.create_fts_index][] for the synchronous API or
+[lancedb.table.AsyncTable.create_index][] with [lancedb.index.FTS][] for the
+asynchronous API.
 
-```python
-from lancedb.index import FTS
-
-table.create_index(
-    "text",
-    config=FTS(remove_stop_words=True, custom_stop_words=["acme", "internal"]),
-)
-```
-
-The list replaces the built-in stop words and is used only when
-`remove_stop_words=True`:
-
-- `custom_stop_words=None` uses the built-in list for `language`.
-- `custom_stop_words=[]` removes no words.
-- Values are passed through without trimming, lowercasing, or other rewriting.
-
-The same option is available on `lancedb.tokenize(...)` and the deprecated
-[lancedb.table.Table.create_fts_index][] compatibility helper:
-
-```python
-import lancedb
-
-tokens = list(lancedb.tokenize("acme makes searchable data",
-                               custom_stop_words=["acme"]))
-```
-
-::: lancedb.tokenize
-
-::: lancedb.FtsToken
-
-## Blobs
-
-Blob columns store large binary values out of line so they can be read lazily
-instead of being materialized with the rest of the row.
-
-::: lancedb.blob
-
-::: lancedb.BlobType
-
-::: lancedb._blob.BlobFile
-    options:
-      show_root_full_path: false
+::: lancedb.index.FTS
 
 ## Utilities
 
@@ -178,14 +106,6 @@ instead of being materialized with the rest of the row.
 
 ::: lancedb.merge.LanceMergeInsertBuilder
 
-::: lancedb.otel.instrument_lancedb_metrics
-
-## Exceptions
-
-::: lancedb.exceptions.MissingValueError
-
-::: lancedb.exceptions.MissingColumnError
-
 ## Integrations
 
 ## Pydantic
@@ -194,30 +114,19 @@ instead of being materialized with the rest of the row.
 
 ::: lancedb.pydantic.vector
 
-::: lancedb.pydantic.Vector
-
-::: lancedb.pydantic.MultiVector
-
 ::: lancedb.pydantic.LanceModel
 
-## PyTorch
-
-::: lancedb.streaming.StreamingDataset
-
-::: lancedb.permutation.permutation_builder
-
-::: lancedb.permutation.PermutationBuilder
-
-::: lancedb.permutation.Permutation
-
-::: lancedb.permutation.Transforms
-
 ## Reranking
 
-::: lancedb.rerankers
-    options:
-      show_root_heading: false
-      show_root_toc_entry: false
+::: lancedb.rerankers.linear_combination.LinearCombinationReranker
+
+::: lancedb.rerankers.cohere.CohereReranker
+
+::: lancedb.rerankers.colbert.ColbertReranker
+
+::: lancedb.rerankers.cross_encoder.CrossEncoderReranker
+
+::: lancedb.rerankers.openai.OpenaiReranker
 
 ## Connections (Asynchronous)
 
@@ -228,12 +137,6 @@ can be used to create, list, or open tables.
 
 ::: lancedb.db.AsyncConnection
 
-## Namespaces (Asynchronous)
-
-::: lancedb.connect_namespace_async
-
-::: lancedb.namespace.AsyncLanceNamespaceDBConnection
-
 ## Tables (Asynchronous)
 
 Table hold your actual data as a collection of records / rows.
@@ -242,20 +145,32 @@ Table hold your actual data as a collection of records / rows.
 
 ::: lancedb.table.AsyncTags
 
-::: lancedb.table.AsyncBranches
-
 ## Indices (Asynchronous)
 
 Indices can be created on a table to speed up queries. This section
 lists the indices that LanceDb supports.
 
-::: lancedb.index
-    options:
-      show_root_heading: false
-      show_root_toc_entry: false
-      # `lang_mapping` is defined in the module rather than imported, so it is
-      # picked up despite not being in `__all__`. It is an internal lookup table.
-      filters: ["!^_", "!^lang_mapping$"]
+::: lancedb.index.BTree
+
+::: lancedb.index.Bitmap
+
+::: lancedb.index.LabelList
+
+::: lancedb.index.FTS
+
+::: lancedb.index.IvfPq
+
+::: lancedb.index.HnswPq
+
+::: lancedb.index.HnswSq
+
+::: lancedb.index.IvfFlat
+
+::: lancedb.index.IvfSq
+
+::: lancedb.index.IvfRq
+
+::: lancedb.index.HnswFlat
 
 ::: lancedb.table.IndexStatistics
 
@@ -283,7 +198,3 @@ rows nearest to a query vector and can be created with the
 ::: lancedb.query.AsyncHybridQuery
     options:
       inherited_members: true
-
-::: lancedb.query.AsyncTakeQuery
-    options:
-      inherited_members: true

java/lancedb-core/pom.xml

@@ -8,7 +8,7 @@
     <parent>
       <groupId>com.lancedb</groupId>
       <artifactId>lancedb-parent</artifactId>
-      <version>0.37.1-final.0</version>
+      <version>0.31.0-beta.6</version>
       <relativePath>../pom.xml</relativePath>
     </parent>
 

java/pom.xml

@@ -6,7 +6,7 @@
 
     <groupId>com.lancedb</groupId>
     <artifactId>lancedb-parent</artifactId>
-    <version>0.37.1-final.0</version>
+    <version>0.31.0-beta.6</version>
     <packaging>pom</packaging>
     <name>${project.artifactId}</name>
     <description>LanceDB Java SDK Parent POM</description>
@@ -28,7 +28,7 @@
     <properties>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
         <arrow.version>15.0.0</arrow.version>
-        <lance-core.version>10.0.0</lance-core.version>
+        <lance-core.version>8.0.0</lance-core.version>
         <spotless.skip>false</spotless.skip>
         <spotless.version>2.30.0</spotless.version>
         <spotless.java.googlejavaformat.version>1.7</spotless.java.googlejavaformat.version>

nodejs/CONTRIBUTING.md

@@ -1,7 +1,7 @@
 # Contributing to LanceDB Typescript
 
 This document outlines the process for contributing to LanceDB Typescript.
-For general contribution guidelines, see [CONTRIBUTING.md](https://github.com/lancedb/lancedb/blob/main/CONTRIBUTING.md).
+For general contribution guidelines, see [CONTRIBUTING.md](../CONTRIBUTING.md).
 
 ## Project layout
 

nodejs/Cargo.toml

@@ -1,7 +1,7 @@
 [package]
 name = "lancedb-nodejs"
 edition.workspace = true
-version = "0.37.1"
+version = "0.31.0-beta.6"
 publish = false
 license.workspace = true
 description.workspace = true
@@ -44,6 +44,6 @@ aws-lc-rs = "=1.16.3"
 napi-build = "2.3.1"
 
 [features]
-default = ["remote", "lancedb/aws", "lancedb/gcs", "lancedb/azure", "lancedb/dynamodb", "lancedb/oss", "lancedb/huggingface", "lancedb/goosefs", "lancedb/metrics-otel"]
+default = ["remote", "lancedb/aws", "lancedb/gcs", "lancedb/azure", "lancedb/dynamodb", "lancedb/oss", "lancedb/huggingface"]
 fp16kernels = ["lancedb/fp16kernels"]
 remote = ["lancedb/remote"]

nodejs/__test__/arrow.test.ts

@@ -6,9 +6,7 @@ import * as arrow17 from "apache-arrow-17";
 import * as arrow18 from "apache-arrow-18";
 
 import {
-  Vector as CurrentVector,
   convertToTable,
-  tableFromIPC as currentTableFromIPC,
   fromBufferToRecordBatch,
   fromDataToBuffer,
   fromRecordBatchToBuffer,
@@ -21,7 +19,6 @@ import {
   FunctionOptions,
 } from "../lancedb/embedding/embedding_function";
 import { EmbeddingFunctionConfig } from "../lancedb/embedding/registry";
-import { sanitizeTable } from "../lancedb/sanitize";
 
 // biome-ignore lint/suspicious/noExplicitAny: skip
 function sampleRecords(): Array<Record<string, any>> {
@@ -55,7 +52,6 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
       Float64,
       Struct,
       List,
-      Map_,
       Int16,
       Int32,
       Int64,
@@ -67,40 +63,12 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
       tableFromIPC,
       DataType,
       Dictionary,
-      RecordBatch: ArrowRecordBatch,
-      Table: ArrowTable,
       Uint8: ArrowUint8,
-      makeData: arrowMakeData,
-      vectorFromArray,
       // biome-ignore lint/suspicious/noExplicitAny: <explanation>
     } = <any>arrow;
     type Schema = ApacheArrow["Schema"];
     type Table = ApacheArrow["Table"];
 
-    function expectValidMapField(
-      // biome-ignore lint/suspicious/noExplicitAny: Arrow Field types vary across supported versions
-      field: any,
-    ): void {
-      expect(DataType.isMap(field.type)).toBe(true);
-      expect(field.type.keysSorted).toBe(true);
-      expect(field.type.children).toHaveLength(1);
-
-      const entries = field.type.children[0];
-      expect(entries.name).toBe("entries");
-      expect(entries.nullable).toBe(false);
-      expect(DataType.isStruct(entries.type)).toBe(true);
-      expect(entries.type.children).toHaveLength(2);
-
-      const [key, value] = entries.type.children;
-      expect([key.name, value.name]).toEqual(["key", "value"]);
-      expect(key.nullable).toBe(false);
-      expect(DataType.isUtf8(key.type)).toBe(true);
-      expect(value.nullable).toBe(true);
-      expect(DataType.isInt(value.type)).toBe(true);
-      expect(value.type.bitWidth).toBe(32);
-      expect(value.type.isSigned).toBe(true);
-    }
-
     // Helper method to verify various ways to create a table
     async function checkTableCreation(
       tableCreationMethod: (
@@ -204,35 +172,6 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
         expect(table.getChild("d")?.toJSON()).toEqual([9n, 10n, null]);
       });
 
-      it("will use a provided FixedSizeList schema with typed array values", function () {
-        const schema = new Schema([
-          new Field("text", new Utf8(), false),
-          new Field(
-            "vector",
-            new FixedSizeList(3, new Field("item", new Float32(), false)),
-            false,
-          ),
-        ]);
-
-        const table = makeArrowTable(
-          [
-            {
-              text: "foo",
-              vector: new Float32Array([1, 2, 3]),
-            },
-          ],
-          { schema },
-        );
-
-        expect(table.getChild("text")?.toJSON()).toEqual(["foo"]);
-        expect(
-          table
-            .getChild("vector")
-            ?.toJSON()
-            .map((value) => value.toJSON()),
-        ).toEqual([[1, 2, 3]]);
-      });
-
       it("will assume the column `vector` is FixedSizeList<Float32> by default", async function () {
         const schema = new Schema([
           new Field("a", new Float(Precision.DOUBLE), true),
@@ -999,176 +938,9 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
           false,
         );
       });
-
-      it("will make an empty table with a Map field", async function () {
-        const schema = new Schema([
-          new Field(
-            "attributes",
-            new Map_(
-              new Field(
-                "entries",
-                new Struct([
-                  new Field("key", new Utf8(), false),
-                  new Field("value", new Int32(), true),
-                ]),
-                false,
-              ),
-              true,
-            ),
-          ),
-        ]);
-
-        const table = makeEmptyTable(schema);
-
-        expectValidMapField(table.schema.fields[0]);
-
-        const buffer = await fromTableToBuffer(table);
-        const roundTripped = tableFromIPC(buffer);
-
-        expectValidMapField(roundTripped.schema.fields[0]);
-      });
-
-      it("preserves string schema metadata", function () {
-        const metadata = new Map([["source", "fixture"]]);
-        const schema = new Schema(
-          [new Field("value", new Int32(), true)],
-          metadata,
-        );
-
-        expect(makeEmptyTable(schema).schema.metadata.get("source")).toBe(
-          "fixture",
-        );
-      });
-
-      it.each([
-        ["non-string keys", new Map<unknown, unknown>([[42, "fixture"]])],
-        ["non-string values", new Map<unknown, unknown>([["source", 42]])],
-        [
-          "non-string keys and values",
-          new Map<unknown, unknown>([[42, false]]),
-        ],
-      ])("rejects schema metadata with %s", function (_, metadataLike) {
-        const metadata = metadataLike as unknown as Map<string, string>;
-        const schema = new Schema(
-          [new Field("value", new Int32(), true)],
-          metadata,
-        );
-
-        expect(() => makeEmptyTable(schema)).toThrow(
-          "Expected metadata, if present, to be a Map<string, string> but it had non-string keys or values",
-        );
-      });
     });
 
     describe("when using two versions of arrow", function () {
-      it("preserves a dictionary shared by multiple fields", async function () {
-        const values = ["alpha", "beta", "alpha"];
-        const dictionaryVector = vectorFromArray(values);
-        const batch = new ArrowRecordBatch({
-          first: dictionaryVector.data[0],
-          second: dictionaryVector.data[0],
-        });
-        const table = new ArrowTable([batch]);
-
-        const sanitized = sanitizeTable(table);
-        expect([...sanitized.getChild("first")!]).toEqual(values);
-        expect([...sanitized.getChild("second")!]).toEqual(values);
-        const firstType = sanitized.schema.fields[0].type as {
-          dictionary: unknown;
-        };
-        const secondType = sanitized.schema.fields[1].type as {
-          dictionary: unknown;
-        };
-        expect(secondType.dictionary).toBe(firstType.dictionary);
-        expect(sanitized.batches[0].data.children[1].dictionary).toBe(
-          sanitized.batches[0].data.children[0].dictionary,
-        );
-
-        const buf = await fromDataToBuffer(table);
-        const actual = currentTableFromIPC(buf);
-        expect([...actual.getChild("first")!]).toEqual(values);
-        expect([...actual.getChild("second")!]).toEqual(values);
-      });
-
-      it("preserves shared dictionary data from another Arrow version", async function () {
-        const values = ["alpha", "beta", "alpha"];
-        const dictionaryVector = vectorFromArray(values);
-        const firstBatch = new ArrowRecordBatch({
-          label: dictionaryVector.slice(0, 2).data[0],
-        });
-        const secondBatch = new ArrowRecordBatch({
-          label: dictionaryVector.slice(2).data[0],
-        });
-        const table = new ArrowTable([firstBatch, secondBatch]);
-
-        const sanitized = sanitizeTable(table);
-        expect([...sanitized.getChild("label")!]).toEqual(values);
-
-        const dictionaries = sanitized.batches.map(
-          (batch) => batch.data.children[0].dictionary,
-        );
-        expect(dictionaries[0]).toBeInstanceOf(CurrentVector);
-        expect(dictionaries[1]).toBe(dictionaries[0]);
-
-        const buf = await fromDataToBuffer(table);
-        const actual = currentTableFromIPC(buf);
-        expect([...actual.getChild("label")!]).toEqual(values);
-      });
-
-      it("preserves shared chunks in growing dictionaries", async function () {
-        const type = new Dictionary(new Utf8(), new Int32(), 42, false);
-        const firstDictionary = vectorFromArray(["alpha", "beta"], new Utf8());
-        const secondDictionary = firstDictionary.concat(
-          vectorFromArray(["gamma"], new Utf8()),
-        );
-        const firstData = arrowMakeData({
-          type,
-          data: Int32Array.from([0, 1]),
-          dictionary: firstDictionary,
-        });
-        const secondData = arrowMakeData({
-          type,
-          data: Int32Array.from([2]),
-          dictionary: secondDictionary,
-        });
-        const table = new ArrowTable([
-          new ArrowRecordBatch({ label: firstData }),
-          new ArrowRecordBatch({ label: secondData }),
-        ]);
-
-        const sanitized = sanitizeTable(table);
-        const expected = ["alpha", "beta", "gamma"];
-        expect([...sanitized.getChild("label")!]).toEqual(expected);
-        const firstLocalDictionary =
-          sanitized.batches[0].data.children[0].dictionary!;
-        const secondLocalDictionary =
-          sanitized.batches[1].data.children[0].dictionary!;
-        expect(secondLocalDictionary.data[0]).toBe(
-          firstLocalDictionary.data[0],
-        );
-
-        const buf = await fromTableToBuffer(sanitized);
-        const actual = currentTableFromIPC(buf);
-        expect([...actual.getChild("label")!]).toEqual(expected);
-      });
-
-      it("can serialize list data from another Arrow version", async function () {
-        const values = [["anime", "action"], [], null];
-        const vector = vectorFromArray(
-          values,
-          new List(new Field("item", new Utf8(), true)),
-        );
-        const table = new ArrowTable({ tags: vector });
-
-        const buf = await fromDataToBuffer(table);
-        const actual = currentTableFromIPC(buf);
-        const actualTags = actual.getChild("tags");
-
-        expect(actualTags?.get(0)?.toJSON()).toEqual(values[0]);
-        expect(actualTags?.get(1)?.toJSON()).toEqual(values[1]);
-        expect(actualTags?.get(2)).toBeNull();
-      });
-
       it("can still import data", async function () {
         const schema = new arrow15.Schema([
           new arrow15.Field("id", new arrow15.Int32()),

nodejs/__test__/embedding.test.ts

@@ -11,11 +11,8 @@ import {
   Float16,
   Float32,
   Float64,
-  Int32,
   Schema,
   Utf8,
-  fromDataToBuffer,
-  tableFromIPC,
 } from "../lancedb/arrow";
 import { EmbeddingFunction, LanceSchema } from "../lancedb/embedding";
 import { getRegistry, register } from "../lancedb/embedding/registry";
@@ -187,63 +184,6 @@ describe("embedding functions", () => {
     const vector0 = JSON.parse(JSON.stringify(arr[0].vector));
     expect(vector0).toEqual([1, 2, 3]);
   });
-
-  it("should append generated vectors to a non-nullable schema", async () => {
-    @register("non_nullable_schema_test")
-    class MockEmbeddingFunction extends EmbeddingFunction<string> {
-      ndims() {
-        return 3;
-      }
-      embeddingDataType(): Float {
-        return new Float64();
-      }
-      async computeSourceEmbeddings(data: string[]) {
-        return data.map(() => [1, 2, 3]);
-      }
-    }
-
-    const schema = new Schema([
-      new Field("id", new Int32()),
-      new Field("text", new Utf8()),
-      new Field("type", new Utf8()),
-      new Field(
-        "vector",
-        new FixedSizeList(3, new Field("item", new Float64())),
-      ),
-    ]);
-    const func = new MockEmbeddingFunction();
-    const db = await connect(tmpDir.name);
-    const table = await db.createEmptyTable("test_non_nullable", schema, {
-      embeddingFunction: {
-        function: func,
-        sourceColumn: "text",
-      },
-    });
-
-    const data = [
-      { id: 1, text: "Carrot", type: "vegetable" },
-      { id: 2, text: "Apple", type: "fruit" },
-    ];
-    const buffer = await fromDataToBuffer(
-      data,
-      undefined,
-      await table.schema(),
-    );
-    const generatedTable = tableFromIPC(buffer);
-    const vectorField = generatedTable.schema.fields.find(
-      (field) => field.name === "vector",
-    );
-    expect(vectorField?.nullable).toBe(false);
-
-    await table.add(data);
-
-    const rows = await table.query().toArray();
-    expect(rows).toHaveLength(2);
-    for (const row of rows) {
-      expect([...row.vector]).toEqual([1, 2, 3]);
-    }
-  });
-
   it("should error when appending to a table with an unregistered embedding function", async () => {
     @register("mock")
     class MockEmbeddingFunction extends EmbeddingFunction<string> {

nodejs/__test__/otel.test.ts

@@ -1,114 +0,0 @@
-// SPDX-License-Identifier: Apache-2.0
-// SPDX-FileCopyrightText: Copyright The LanceDB Authors
-
-import {
-  MeterProvider,
-  type MetricData,
-  MetricReader,
-} from "@opentelemetry/sdk-metrics";
-import * as tmp from "tmp";
-import { connect, instrumentLanceDbMetrics } from "../lancedb";
-// snapshotLancedbMetrics is internal plumbing (not part of the public API), so
-// it is imported from the native module rather than the package entry point.
-import { snapshotLancedbMetrics } from "../lancedb/native";
-
-// The metrics recorder is process-global and installed once, so the whole
-// bridge is exercised in a single test to avoid cross-test global-state coupling.
-
-// A minimal pull-based reader whose `collect()` we drive directly, invoking the
-// observable-instrument callbacks. `@opentelemetry/sdk-metrics` ships no
-// in-memory reader, so we subclass the abstract base.
-class TestMetricReader extends MetricReader {
-  protected async onForceFlush(): Promise<void> {
-    // no-op: collection is driven directly via collect()
-  }
-  protected async onShutdown(): Promise<void> {
-    // no-op: nothing to release
-  }
-}
-
-async function metricsByName(
-  reader: TestMetricReader,
-): Promise<Map<string, MetricData>> {
-  const collected = await reader.collect();
-  const result = new Map<string, MetricData>();
-  for (const scope of collected.resourceMetrics.scopeMetrics) {
-    for (const metric of scope.metrics) {
-      result.set(metric.descriptor.name, metric);
-    }
-  }
-  return result;
-}
-
-describe("OpenTelemetry metrics bridge", () => {
-  let tmpDir: tmp.DirResult;
-  beforeEach(() => {
-    tmpDir = tmp.dirSync({ unsafeCleanup: true });
-  });
-  afterEach(() => tmpDir.removeCallback());
-
-  it("snapshot is safe to call regardless of install state", () => {
-    expect(Array.isArray(snapshotLancedbMetrics())).toBe(true);
-  });
-
-  it("exports object store metrics via observable instruments", async () => {
-    const reader = new TestMetricReader();
-    const provider = new MeterProvider({ readers: [reader] });
-    expect(instrumentLanceDbMetrics(provider)).toBe(true);
-
-    // Generate object store activity on the local filesystem (scheme "file").
-    const db = await connect(tmpDir.name);
-    const data = Array.from({ length: 256 }, (_, i) => ({ id: i }));
-    const table = await db.createTable("t", data);
-    expect(await table.countRows()).toBe(256);
-
-    const metrics = await metricsByName(reader);
-
-    const requests = metrics.get("lance_object_store_requests_total");
-    expect(requests).toBeDefined();
-    // biome-ignore lint/suspicious/noExplicitAny: SDK point shape
-    const requestPoints = (requests!.dataPoints as any[]) ?? [];
-    expect(requestPoints.length).toBeGreaterThan(0);
-    for (const p of requestPoints) {
-      // Labelled by `operation` and `base` (the store scheme by default).
-      expect(p.attributes).toHaveProperty("base");
-      expect(p.attributes).toHaveProperty("operation");
-    }
-    const totalRequests = requestPoints.reduce((acc, p) => acc + p.value, 0);
-    expect(totalRequests).toBeGreaterThan(0);
-
-    // Histograms are decomposed into bucket / count / sum observable counters.
-    const bucket = metrics.get(
-      "lance_object_store_request_duration_seconds_bucket",
-    );
-    expect(bucket).toBeDefined();
-    // biome-ignore lint/suspicious/noExplicitAny: SDK point shape
-    const bucketPoints = (bucket!.dataPoints as any[]) ?? [];
-    expect(bucketPoints.length).toBeGreaterThan(0);
-    expect(bucketPoints.every((p) => "le" in p.attributes)).toBe(true);
-    // The implicit +Inf bucket must be present.
-    expect(bucketPoints.some((p) => p.attributes.le === "+Inf")).toBe(true);
-
-    const count = metrics.get(
-      "lance_object_store_request_duration_seconds_count",
-    );
-    expect(count).toBeDefined();
-    // biome-ignore lint/suspicious/noExplicitAny: SDK point shape
-    const countPoints = (count!.dataPoints as any[]) ?? [];
-    expect(countPoints.reduce((acc, p) => acc + p.value, 0)).toBeGreaterThan(0);
-
-    const sum = metrics.get("lance_object_store_request_duration_seconds_sum");
-    expect(sum).toBeDefined();
-    // biome-ignore lint/suspicious/noExplicitAny: SDK point shape
-    const sumPoints = (sum!.dataPoints as any[]) ?? [];
-    expect(sumPoints.reduce((acc, p) => acc + p.value, 0)).toBeGreaterThan(0);
-
-    // Unit handling: only `_sum` keeps the histogram's unit (seconds); `_bucket`
-    // and `_count` observe cumulative counts and are unitless.
-    expect(sum!.descriptor.unit).toBe("s");
-    expect(bucket!.descriptor.unit).toBe("");
-    expect(count!.descriptor.unit).toBe("");
-
-    await provider.shutdown();
-  });
-});

nodejs/__test__/package.test.ts

@@ -1,14 +0,0 @@
-// SPDX-License-Identifier: Apache-2.0
-// SPDX-FileCopyrightText: Copyright The LanceDB Authors
-
-import packageJson = require("../package.json");
-
-describe("package metadata", () => {
-  it("requires Node.js type declarations compatible with the runtime", () => {
-    expect(packageJson.engines.node).toBe(">= 18");
-    expect(packageJson.peerDependencies["@types/node"]).toBe(">=18");
-    expect(packageJson.peerDependenciesMeta["@types/node"]).toEqual({
-      optional: true,
-    });
-  });
-});

nodejs/__test__/query.test.ts

@@ -110,81 +110,6 @@ describe("Query outputSchema", () => {
   });
 });
 
-describe("Search pagination", () => {
-  let tmpDir: tmp.DirResult;
-  let table: Table;
-
-  beforeEach(async () => {
-    tmpDir = tmp.dirSync({ unsafeCleanup: true });
-    const db = await connect(tmpDir.name);
-    const schema = new Schema([
-      new Field("id", new Int64(), false),
-      new Field("text", new Utf8(), false),
-      new Field(
-        "vector",
-        new FixedSizeList(2, new Field("item", new Float32())),
-        false,
-      ),
-    ]);
-    const data = makeArrowTable(
-      [
-        { id: 1n, text: "common", vector: [0, 0] },
-        { id: 2n, text: "common common", vector: [1, 1] },
-        { id: 3n, text: "common common common", vector: [2, 2] },
-        { id: 4n, text: "common common common common", vector: [3, 3] },
-      ],
-      { schema },
-    );
-    table = await db.createTable("test", data);
-  });
-
-  afterEach(() => {
-    tmpDir.removeCallback();
-  });
-
-  it("applies offset after the vector search limit", async () => {
-    const allResults = await table
-      .vectorSearch([0, 0])
-      .select(["id"])
-      .limit(4)
-      .toArray();
-    const secondPage = await table
-      .vectorSearch([0, 0])
-      .select(["id"])
-      .limit(2)
-      .offset(2)
-      .toArray();
-
-    expect(allResults).toHaveLength(4);
-    expect(secondPage).toHaveLength(2);
-    expect(secondPage.map((row) => row.id)).toEqual(
-      allResults.slice(2, 4).map((row) => row.id),
-    );
-  });
-
-  it("applies offset after the full-text search limit", async () => {
-    await table.createIndex("text", { config: Index.fts() });
-
-    const allResults = await table
-      .search("common", "fts")
-      .select(["id"])
-      .limit(4)
-      .toArray();
-    const secondPage = await table
-      .search("common", "fts")
-      .select(["id"])
-      .limit(2)
-      .offset(2)
-      .toArray();
-
-    expect(allResults).toHaveLength(4);
-    expect(secondPage).toHaveLength(2);
-    expect(secondPage.map((row) => row.id)).toEqual(
-      allResults.slice(2, 4).map((row) => row.id),
-    );
-  });
-});
-
 describe("Query orderBy", () => {
   let tmpDir: tmp.DirResult;
   let table: Table;

nodejs/__test__/remote.test.ts

@@ -15,7 +15,6 @@ import {
   OAuthHeaderProvider,
   StaticHeaderProvider,
 } from "../lancedb/header";
-import { Index } from "../lancedb/indices";
 
 // Test-only header providers
 class CustomProvider extends HeaderProvider {
@@ -170,38 +169,6 @@ describe("remote connection", () => {
     );
   });
 
-  it("surfaces JSON server errors from remote table operations", async () => {
-    await withMockDatabase(
-      (req, res) => {
-        const path = req.url ?? "";
-        if (path.endsWith("/describe/")) {
-          res.writeHead(200, { "Content-Type": "application/json" }).end(
-            JSON.stringify({
-              name: "broken_table",
-              version: 1,
-              schema: { fields: [] },
-            }),
-          );
-          return;
-        }
-
-        if (path.endsWith("/count_rows/")) {
-          res
-            .writeHead(400, { "Content-Type": "application/json" })
-            .end(JSON.stringify({ error: "count rows failed" }));
-          return;
-        }
-
-        res.writeHead(404).end();
-      },
-      async (db) => {
-        const table = await db.openTable("broken_table");
-
-        await expect(table.countRows()).rejects.toThrow("count rows failed");
-      },
-    );
-  });
-
   it("should pass on requested extra headers", async () => {
     await withMockDatabase(
       (req, res) => {
@@ -258,166 +225,6 @@ describe("remote connection", () => {
     );
   });
 
-  it("sends FTS options to remote tables", async () => {
-    let createIndexBody: Record<string, unknown> | undefined;
-
-    await withMockDatabase(
-      (req, res) => {
-        const path = req.url ?? "";
-        if (path.endsWith("/describe/")) {
-          res.writeHead(200, { "Content-Type": "application/json" }).end(
-            JSON.stringify({
-              name: "t",
-              version: 1,
-              schema: {
-                fields: [
-                  { name: "text", type: { type: "string" }, nullable: false },
-                ],
-              },
-            }),
-          );
-          return;
-        }
-
-        if (path.endsWith("/create_index/")) {
-          let raw = "";
-          req.on("data", (chunk) => {
-            raw += chunk;
-          });
-          req.on("end", () => {
-            createIndexBody = JSON.parse(raw);
-            res.writeHead(200).end();
-          });
-          return;
-        }
-
-        res.writeHead(404).end();
-      },
-      async (db) => {
-        const table = await db.openTable("t");
-        await table.createIndex("text", {
-          config: Index.fts({
-            blockSize: 256,
-            removeStopWords: true,
-            customStopWords: ["the"],
-          }),
-        });
-      },
-    );
-
-    expect(createIndexBody?.["column"]).toBe("text");
-    expect(createIndexBody?.["index_type"]).toBe("FTS");
-    expect(createIndexBody?.["block_size"]).toBe(256);
-    expect(createIndexBody?.["custom_stop_words"]).toEqual(["the"]);
-  });
-
-  it("diffs and merges remote branches", async () => {
-    const sampleDiff = {
-      fromBranch: "exp",
-      parentVersion: 1,
-      mainVersion: 2,
-      branchVersion: 3,
-      baseMoved: false,
-      rowCountMain: 3,
-      rowCountBranch: 3,
-      rowSummary: {
-        unchanged: 3,
-        newOnBase: 0,
-        newOnBranch: 0,
-        staleRecompute: 0,
-        inputsChanged: 0,
-        deltaAvailable: false,
-      },
-      addedColumns: [{ name: "tag", dataType: "utf8", nullable: true }],
-      removedColumns: [],
-      changedColumns: [],
-      addedIndexes: [],
-      removedIndexes: [],
-      mergeable: true,
-      mergeBlockers: [],
-    };
-    const mergeBodies: Record<string, unknown>[] = [];
-
-    await withMockDatabase(
-      (req, res) => {
-        const path = req.url ?? "";
-        if (path.endsWith("/describe/")) {
-          res.writeHead(200, { "Content-Type": "application/json" }).end(
-            JSON.stringify({
-              name: "t",
-              version: 2,
-              schema: { fields: [] },
-            }),
-          );
-          return;
-        }
-
-        let raw = "";
-        req.on("data", (chunk) => {
-          raw += chunk;
-        });
-        req.on("end", () => {
-          const body = raw ? JSON.parse(raw) : {};
-          if (path.endsWith("/branches/diff/")) {
-            // biome-ignore lint/style/useNamingConvention: snake_case mandated by the server wire format
-            expect(body).toEqual({ from_branch: "exp" });
-            res
-              .writeHead(200, { "Content-Type": "application/json" })
-              .end(JSON.stringify(sampleDiff));
-            return;
-          }
-          if (path.endsWith("/branches/merge/")) {
-            mergeBodies.push(body);
-            const dryRun = body["dry_run"] === true;
-            const response = {
-              status: dryRun ? "ready" : "rejected",
-              diff: dryRun
-                ? sampleDiff
-                : {
-                    ...sampleDiff,
-                    mergeable: false,
-                    mergeBlockers: [
-                      { code: "baseMoved", message: "main has advanced" },
-                    ],
-                  },
-              preview: { promotedColumns: dryRun ? ["tag"] : [] },
-            };
-            res
-              .writeHead(dryRun ? 200 : 409, {
-                "Content-Type": "application/json",
-              })
-              .end(JSON.stringify(response));
-            return;
-          }
-          res.writeHead(404).end();
-        });
-      },
-      async (db) => {
-        const table = await db.openTable("t");
-        const branches = await table.branches();
-
-        await expect(branches.diff("exp")).resolves.toEqual(sampleDiff);
-
-        const rejected = await branches.merge("exp");
-        expect(rejected.status).toBe("rejected");
-        expect(rejected.diff.mergeBlockers).toEqual([
-          { code: "baseMoved", message: "main has advanced" },
-        ]);
-
-        const preview = await branches.merge("exp", true);
-        expect(preview.status).toBe("ready");
-        expect(preview.preview.promotedColumns).toEqual(["tag"]);
-      },
-    );
-
-    expect(mergeBodies).toEqual([
-      // biome-ignore lint/style/useNamingConvention: snake_case mandated by the server wire format
-      { from_branch: "exp", dry_run: false },
-      // biome-ignore lint/style/useNamingConvention: snake_case mandated by the server wire format
-      { from_branch: "exp", dry_run: true },
-    ]);
-  });
-
   describe("TlsConfig", () => {
     it("should create TlsConfig with all fields", () => {
       const tlsConfig: TlsConfig = {
@@ -909,96 +716,3 @@ describe("remote connection", () => {
     });
   });
 });
-
-describe("remote connection jobs surface", () => {
-  it("lists, describes, cancels, and reads history", async () => {
-    const { tableFromArrays, tableToIPC } = await import("apache-arrow");
-    const eventsTable = tableFromArrays({ state: ["created", "succeeded"] });
-    const eventsBody = Buffer.from(tableToIPC(eventsTable, "stream"));
-
-    await withMockDatabase(
-      (req, res) => {
-        let body = "";
-        req.on("data", (chunk) => {
-          body += chunk;
-        });
-        req.on("end", () => {
-          const payload = body.length > 0 ? JSON.parse(body) : {};
-          if (req.url === "/v1/jobs/list") {
-            if (payload["page_token"] === undefined) {
-              res
-                .writeHead(200, { "Content-Type": "application/json" })
-                .end(
-                  '{"jobs": [{"job_id": "job-1", "table": "t1", ' +
-                    '"job_type": "create_index", "state": "in_progress", ' +
-                    '"created_at_millis": 1000}], "page_token": "next"}',
-                );
-            } else {
-              res
-                .writeHead(200, { "Content-Type": "application/json" })
-                .end(
-                  '{"jobs": [{"job_id": "job-2", "table": "t2", ' +
-                    '"job_type": "create_index", "state": "succeeded", ' +
-                    '"created_at_millis": 2000}]}',
-                );
-            }
-          } else if (req.url === "/v1/jobs/describe") {
-            if (payload["job_id"] !== "job-1") {
-              res.writeHead(404).end("no such job");
-              return;
-            }
-            res
-              .writeHead(200, { "Content-Type": "application/json" })
-              .end(
-                '{"job_id": "job-1", "job_type": "create_index", ' +
-                  '"job_state": "FAILED", "creation_ms": 1000, ' +
-                  '"spec": {"column": "vec"}, "failure": {"phase": "execute", ' +
-                  '"message": "worker died", "retryable": true}}',
-              );
-          } else if (req.url === "/v1/jobs/cancel") {
-            if (payload["job_id"] !== "job-1") {
-              res.writeHead(404).end("no such job");
-              return;
-            }
-            res
-              .writeHead(200, { "Content-Type": "application/json" })
-              .end('{"job_id": "job-1"}');
-          } else if (req.url === "/v1/jobs/query_events") {
-            res
-              .writeHead(200, {
-                "Content-Type": "application/vnd.apache.arrow.stream",
-              })
-              .end(eventsBody);
-          } else {
-            res.writeHead(404).end();
-          }
-        });
-      },
-      async (db) => {
-        const jobs = await db.listJobs();
-        expect(jobs.map((job) => job.jobId)).toEqual(["job-1", "job-2"]);
-        expect(jobs[0].state).toEqual("running");
-        expect(jobs[1].state).toEqual("finished");
-
-        const description = await db.getJob("job-1");
-        expect(description?.state).toEqual("failed");
-        expect(JSON.parse(description?.specJson ?? "")).toEqual({
-          column: "vec",
-        });
-        expect(description?.failure?.message).toEqual("worker died");
-        expect(await db.getJob("missing")).toBeNull();
-
-        expect(await db.cancelJob("job-1")).toBe(true);
-        expect(await db.cancelJob("missing")).toBe(false);
-
-        const history = await db.jobHistory("job-1");
-        expect(history.numRows).toEqual(2);
-
-        const job = db.job("job-1");
-        expect(job.id).toEqual("job-1");
-        expect(await job.status()).toEqual("failed");
-        await expect(job.wait()).rejects.toThrow("worker died");
-      },
-    );
-  });
-});

nodejs/__test__/sanitize.test.ts

@@ -2,7 +2,7 @@
 // SPDX-FileCopyrightText: Copyright The LanceDB Authors
 
 import * as arrow from "../lancedb/arrow";
-import { sanitizeField, sanitizeMap, sanitizeType } from "../lancedb/sanitize";
+import { sanitizeField, sanitizeType } from "../lancedb/sanitize";
 
 describe("sanitize", function () {
   describe("sanitizeType function", function () {
@@ -181,15 +181,4 @@ describe("sanitize", function () {
       );
     });
   });
-
-  describe("sanitizeMap function", function () {
-    it.each([
-      ["no children", []],
-      ["two children", [{}, {}]],
-    ])("should reject a Map type with %s", function (_, children) {
-      expect(() => sanitizeMap({ children, keysSorted: false })).toThrow(
-        "Expected a Map type to have exactly one child",
-      );
-    });
-  });
 });

nodejs/__test__/table.test.ts

@@ -16,7 +16,6 @@ import {
   PhraseQuery,
   Table,
   connect,
-  tokenize,
 } from "../lancedb";
 import {
   Table as ArrowTable,
@@ -86,44 +85,6 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
       await expect(table.countRows()).resolves.toBe(3);
     });
 
-    it("should support a foreign Float64 vector schema end to end", async () => {
-      const conn = await connect(tmpDir.name);
-      const schema = new arrow.Schema([
-        new arrow.Field("resource_id", new arrow.Int32(), false),
-        new arrow.Field(
-          "vector",
-          new arrow.FixedSizeList(
-            3,
-            new arrow.Field("value", new arrow.Float64(), true),
-          ),
-          false,
-        ),
-      ]);
-      const data = [
-        {
-          // biome-ignore lint/style/useNamingConvention: matches the reported schema
-          resource_id: 0,
-          vector: [0.1, 0.1, 0.1],
-        },
-      ];
-
-      const resources = await conn.createTable("resources", data, { schema });
-
-      const existing = await resources
-        .query()
-        .where("resource_id = 0")
-        .limit(1)
-        .toArray();
-      expect(existing).toHaveLength(1);
-
-      const matched = await resources
-        .search(Float64Array.from(data[0].vector))
-        .limit(1)
-        .toArray();
-      expect(matched).toHaveLength(1);
-      expect(matched[0]["resource_id"]).toBe(0);
-    });
-
     it("should support branches", async () => {
       await table.add([{ id: 1 }]);
       expect(await table.countRows()).toBe(1);
@@ -277,16 +238,8 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
         },
         numIndices: 0,
         numRows: 3,
-        // Full on-disk size of the two data files, footers and metadata included.
-        totalBytes: 684,
+        totalBytes: 44,
       });
-
-      // Index files count toward totalBytes too (only deletion files and
-      // manifests are excluded).
-      await table.createIndex("id", { config: Index.btree() });
-      const statsWithIndex = await table.stats();
-      expect(statsWithIndex.numIndices).toBe(1);
-      expect(statsWithIndex.totalBytes).toBeGreaterThan(684);
     });
 
     it("should overwrite data if asked", async () => {
@@ -573,14 +526,6 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
       );
     });
 
-    it("should expose useLsm on takeRowIds as the base-only escape hatch", async () => {
-      await table.add([{ id: 1 }, { id: 2 }, { id: 3 }]);
-      // useLsm(false) is reachable on TakeQuery (the escape hatch for MemWAL tables,
-      // where take-by-row-id auto-routes to the LSM scanner and is rejected).
-      const res = await table.takeRowIds([0, 2]).useLsm(false).toArray();
-      expect(res.map((r) => r.id)).toEqual([1, 3]);
-    });
-
     it("should throw for negative number in takeRowIds", () => {
       expect(() => table.takeRowIds([-1])).toThrow("Row id cannot be negative");
       expect(() => table.takeRowIds([0, -5, 2])).toThrow(
@@ -897,11 +842,7 @@ describe("When creating an index", () => {
   afterEach(() => tmpDir.removeCallback());
 
   it("should create a vector index on vector columns", async () => {
-    const job = await tbl.createIndexAsync("vec");
-    expect(job.id).toBeNull();
-    await job.wait();
-    // Cancelling a job that already finished succeeds and does nothing.
-    await job.cancel();
+    await tbl.createIndex("vec");
 
     // check index directory
     const indexDir = path.join(tmpDir.name, "test.lance", "_indices");
@@ -2366,75 +2307,6 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
       expect(results2[0].text).toBe(data[1].text);
     });
 
-    test("tokenizes FTS queries by column or index name", async () => {
-      const db = await connect(tmpDir.name);
-      const data = [
-        {
-          text: "Running in cafés",
-          japanese: "Hello, こんにちは世界!",
-          vector: [0.1, 0.2, 0.3],
-        },
-      ];
-      const table = await db.createTable("test", data);
-      await table.createIndex("text", {
-        config: Index.fts({ baseTokenizer: "simple" }),
-      });
-      await table.createIndex("japanese", {
-        config: Index.fts({
-          baseTokenizer: "icu",
-          stem: false,
-          removeStopWords: false,
-        }),
-        name: "japanese_icu_idx",
-      });
-
-      await expect(table.tokenize("hello", {} as never)).rejects.toThrow(
-        "Specify exactly one",
-      );
-      await expect(
-        table.tokenize("hello", {
-          column: "text",
-          indexName: "text_idx",
-        } as never),
-      ).rejects.toThrow("Specify exactly one");
-
-      const simpleTokens = await table.tokenize("Running in cafés", {
-        column: "text",
-      });
-      expect(simpleTokens).toEqual([
-        { text: "run", position: 0 },
-        { text: "cafe", position: 2 },
-      ]);
-
-      const icuTokens = await table.tokenize("Hello, こんにちは世界!", {
-        indexName: "japanese_icu_idx",
-      });
-      expect(icuTokens).toEqual([
-        { text: "hello", position: 0 },
-        { text: "こんにちは", position: 1 },
-        { text: "世界", position: 2 },
-      ]);
-
-      const directSimpleTokens = await tokenize("Running in cafés", {
-        baseTokenizer: "simple",
-      });
-      expect(directSimpleTokens).toEqual([
-        { text: "run", position: 0 },
-        { text: "cafe", position: 2 },
-      ]);
-
-      const directIcuTokens = await tokenize("Hello, こんにちは世界!", {
-        baseTokenizer: "icu",
-        stem: false,
-        removeStopWords: false,
-      });
-      expect(directIcuTokens).toEqual([
-        { text: "hello", position: 0 },
-        { text: "こんにちは", position: 1 },
-        { text: "世界", position: 2 },
-      ]);
-    });
-
     test("full text search fast search", async () => {
       const db = await connect(tmpDir.name);
       const data = [{ text: "hello world", vector: [0.1, 0.2, 0.3], id: 1 }];
@@ -2585,35 +2457,6 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
       expect(results3.length).toBe(1);
     });
 
-    test("full text search with custom posting block size", async () => {
-      const db = await connect(tmpDir.name);
-      const data = [
-        { text: "hello world", vector: [0.1, 0.2, 0.3] },
-        { text: "goodbye world", vector: [0.4, 0.5, 0.6] },
-      ];
-      const table = await db.createTable("test", data);
-      await table.createIndex("text", {
-        config: Index.fts({ blockSize: 256 }),
-      });
-
-      const index = (await table.listIndices()).find(
-        (index) => index.indexType === "FTS",
-      );
-      expect(index?.indexVersion).toBe(3);
-      expect(
-        (index?.indexDetails as Record<string, unknown>)["block_size"],
-      ).toBe(256);
-
-      const results = await table.search("hello").toArray();
-      expect(results[0].text).toBe(data[0].text);
-    });
-
-    test("rejects invalid full text posting block size", () => {
-      expect(() => Index.fts({ blockSize: 129 as 128 | 256 })).toThrow(
-        "128 or 256",
-      );
-    });
-
     test("full text search without lowercase", async () => {
       const db = await connect(tmpDir.name);
       const data = [
@@ -2819,15 +2662,6 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
   },
 );
 
-test("tokenize supports custom stop words", async () => {
-  const tokens = await tokenize("the lance data", {
-    stem: false,
-    removeStopWords: true,
-    customStopWords: ["lance"],
-  });
-  expect(tokens.map((token) => token.text)).toEqual(["the", "data"]);
-});
-
 describe("when calling explainPlan", () => {
   let tmpDir: tmp.DirResult;
   let table: Table;
@@ -2871,13 +2705,8 @@ describe("when calling analyzePlan", () => {
       .fill(1)
       .map(() => Math.random());
     const plan = await table.query().nearestTo(queryVec).analyzePlan();
+    console.log("Query Plan:\n", plan); // <--- Print the plan
     expect(plan).toMatch("AnalyzeExec");
-
-    const fullPlan = await table
-      .query()
-      .nearestTo(queryVec)
-      .analyzePlan("full");
-    expect(fullPlan).toMatch("AnalyzeExec");
   });
 });
 
@@ -3163,56 +2992,6 @@ describe("setLsmWriteSpec / unsetLsmWriteSpec", () => {
       }),
     ).rejects.toThrow();
   });
-
-  it("reads back the installed spec via getLsmWriteSpec", async () => {
-    const conn = await connect(tmpDir.name);
-    const table = await makeTable(conn);
-    await table.setUnenforcedPrimaryKey("id");
-
-    // Nothing installed yet.
-    expect(await table.getLsmWriteSpec()).toBeUndefined();
-
-    // A real scalar index is needed to name it as a maintained index.
-    await table.add([{ id: 1 }, { id: 2 }, { id: 3 }]);
-    await table.createIndex("id");
-    const indexName = (await table.listIndices())[0].name;
-
-    // Bucket spec round-trips, including maintained indexes and writer config
-    // defaults. Lance writer-config keys are canonically snake_case.
-    // biome-ignore lint/style/useNamingConvention: Lance writer-config keys are snake_case
-    const writerConfigDefaults = { durable_write: "false" };
-    await table.setLsmWriteSpec({
-      specType: "bucket",
-      column: "id",
-      numBuckets: 4,
-      maintainedIndexes: [indexName],
-      writerConfigDefaults,
-    });
-    const spec = await table.getLsmWriteSpec();
-    expect(spec).toBeDefined();
-    expect(spec?.specType).toBe("bucket");
-    expect(spec?.column).toBe("id");
-    expect(spec?.numBuckets).toBe(4);
-    expect(spec?.maintainedIndexes).toEqual([indexName]);
-    expect(spec?.writerConfigDefaults).toEqual(writerConfigDefaults);
-
-    // After unset, undefined again.
-    await table.unsetLsmWriteSpec();
-    expect(await table.getLsmWriteSpec()).toBeUndefined();
-
-    // Identity round-trips (column recovered from the schema).
-    await table.setLsmWriteSpec({ specType: "identity", column: "id" });
-    const identity = await table.getLsmWriteSpec();
-    expect(identity?.specType).toBe("identity");
-    expect(identity?.column).toBe("id");
-    await table.unsetLsmWriteSpec();
-
-    // Unsharded round-trips (no routing column).
-    await table.setLsmWriteSpec({ specType: "unsharded" });
-    const unsharded = await table.getLsmWriteSpec();
-    expect(unsharded?.specType).toBe("unsharded");
-    expect(unsharded?.column).toBeFalsy();
-  });
 });
 
 describe("LSM merge insert", () => {
@@ -3266,14 +3045,14 @@ describe("LSM merge insert", () => {
     await table.closeLsmWriters();
   });
 
-  it("falls back to the standard path with useLsm(false)", async () => {
+  it("falls back to the standard path with useLsmWrite(false)", async () => {
     const conn = await connect(tmpDir.name);
     const table = await bucketTable(conn);
 
     const res = await table
       .mergeInsert("id")
       .whenNotMatchedInsertAll()
-      .useLsm(false)
+      .useLsmWrite(false)
       .execute([
         { id: "b", value: 9 },
         { id: "e", value: 5 },
@@ -3307,36 +3086,4 @@ describe("LSM merge insert", () => {
         .execute([{ id: "g", value: 7 }]),
     ).rejects.toThrow();
   });
-
-  it("auto-routes reads through the MemWAL scanner", async () => {
-    const conn = await connect(tmpDir.name);
-    const table = await bucketTable(conn); // base ids "a", "b"
-
-    await table
-      .mergeInsert("id")
-      .whenMatchedUpdateAll()
-      .whenNotMatchedInsertAll()
-      .execute([{ id: "c", value: 3 }]);
-
-    // Default read auto-routes and includes the active memtable row.
-    const lsm = await table.query().toArray();
-    expect(lsm.map((r) => r.id).sort()).toEqual(["a", "b", "c"]);
-
-    // useLsm(false) bypasses the MemWAL and reads the base table only.
-    const baseOnly = await table.query().useLsm(false).toArray();
-    expect(baseOnly.map((r) => r.id).sort()).toEqual(["a", "b"]);
-  });
-
-  it("reads the base table when no LSM spec is installed", async () => {
-    const conn = await connect(tmpDir.name);
-    const table = await conn.createEmptyTable(
-      "plain",
-      new arrow.Schema([new arrow.Field("id", new arrow.Utf8(), false)]),
-    );
-    // No spec: default read and useLsm(false) both succeed against the base table.
-    await expect(table.query().toArray()).resolves.toBeDefined();
-    await expect(table.query().useLsm(false).toArray()).resolves.toBeDefined();
-    // useLsm(true) demands MemWAL routing; without a spec it errors.
-    await expect(table.query().useLsm(true).toArray()).rejects.toThrow();
-  });
 });

nodejs/examples/full_text_search.test.ts

@@ -29,14 +29,8 @@ test("full text search", async () => {
     const tbl = await db.createTable("myVectors", data, { mode: "overwrite" });
 
     await tbl.createIndex("doc", {
-      config: lancedb.Index.fts({
-        stem: false,
-        removeStopWords: true,
-        customStopWords: ["banana"],
-      }),
+      config: lancedb.Index.fts(),
     });
-    const tokens = await tbl.tokenize("apple banana", { column: "doc" });
-    expect(tokens.map((token) => token.text)).toEqual(["apple"]);
 
     // --8<-- [start:full_text_search]
     const result = await tbl

nodejs/lancedb/connection.ts

@@ -1,7 +1,6 @@
 // SPDX-License-Identifier: Apache-2.0
 // SPDX-FileCopyrightText: Copyright The LanceDB Authors
 
-import { tableFromIPC } from "apache-arrow";
 import {
   Data,
   SchemaLike,
@@ -21,9 +20,6 @@ import type {
   CreateNamespaceResponse,
   DescribeNamespaceResponse,
   DropNamespaceResponse,
-  Job,
-  JobDescription,
-  JobInfo,
   ListNamespacesResponse,
 } from "./native";
 export type {
@@ -440,40 +436,6 @@ export abstract class Connection {
     newName: string,
     options?: RenameTableOptions,
   ): Promise<void>;
-
-  /**
-   * A {@link Job} handle for a server-side job by id.
-   *
-   * The handle is constructed without a server round trip; an unknown id
-   * surfaces when the handle is used. Dropping the handle has no effect on
-   * the job itself.
-   */
-  abstract job(jobId: string): Job;
-
-  /** List server-side jobs across the database's tables. */
-  abstract listJobs(): Promise<JobInfo[]>;
-
-  /**
-   * Describe a single server-side job by id.
-   *
-   * Resolves to `null` when the server has no such job.
-   */
-  abstract getJob(jobId: string): Promise<JobDescription | null>;
-
-  /**
-   * Request cancellation of a server-side job by id.
-   *
-   * Resolves to true if the server accepted the cancellation, false if no
-   * such job exists. Cancelling an already-terminal job is a no-op success.
-   */
-  abstract cancelJob(jobId: string): Promise<boolean>;
-
-  /**
-   * The lifecycle event history of a server-side job, as an Arrow table.
-   *
-   * Lists history across all jobs when `jobId` is omitted.
-   */
-  abstract jobHistory(jobId?: string): Promise<ArrowTable>;
 }
 
 /** @hideconstructor */
@@ -760,30 +722,6 @@ export class LocalConnection extends Connection {
       options?.newNamespacePath,
     );
   }
-
-  job(jobId: string): Job {
-    return this.inner.job(jobId);
-  }
-
-  async listJobs(): Promise<JobInfo[]> {
-    return this.inner.listJobs();
-  }
-
-  async getJob(jobId: string): Promise<JobDescription | null> {
-    return this.inner.getJob(jobId);
-  }
-
-  async cancelJob(jobId: string): Promise<boolean> {
-    return this.inner.cancelJob(jobId);
-  }
-
-  async jobHistory(jobId?: string): Promise<ArrowTable> {
-    const buf = await this.inner.jobHistory(jobId);
-    if (buf.length === 0) {
-      return new ArrowTable();
-    }
-    return tableFromIPC(buf);
-  }
 }
 
 /**

nodejs/lancedb/index.ts

@@ -13,21 +13,13 @@ import {
   Connection as LanceDbConnection,
   JsHeaderProvider as NativeJsHeaderProvider,
   Session,
-  tokenize as nativeTokenize,
 } from "./native.js";
 
 import { HeaderProvider } from "./header";
-import type { BaseTokenizer } from "./indices";
-import type { FtsToken } from "./table";
 
 // Re-export native header provider for use with connectWithHeaderProvider
 export { JsHeaderProvider as NativeJsHeaderProvider } from "./native.js";
 
-// OpenTelemetry metrics bridge. Only the high-level entry point is public; the
-// underlying recorder/catalog/snapshot functions remain internal plumbing that
-// `otel.ts` consumes from the native module.
-export { instrumentLanceDbMetrics } from "./otel";
-
 export {
   AddColumnsSql,
   ConnectionOptions,
@@ -85,13 +77,7 @@ export {
   RenameTableOptions,
 } from "./connection";
 
-export {
-  Job,
-  JobDescription,
-  JobFailureInfo,
-  JobInfo,
-  Session,
-} from "./native.js";
+export { Session } from "./native.js";
 
 export {
   ExecutableQuery,
@@ -99,7 +85,6 @@ export {
   QueryBase,
   VectorQuery,
   TakeQuery,
-  AnalyzePlanDistributedMetrics,
   QueryExecutionOptions,
   ColumnOrdering,
   FullTextSearchOptions,
@@ -124,27 +109,16 @@ export {
   HnswPqOptions,
   HnswSqOptions,
   FtsOptions,
-  BaseTokenizer,
 } from "./indices";
 
 export {
   Table,
   Branches,
-  BranchColumnSummary,
-  BranchColumnChange,
-  BranchIndexSummary,
-  BranchRowCountSummary,
-  MergeBlocker,
-  BranchDiff,
-  MergePreview,
-  MergeBranchResult,
   AddDataOptions,
   UpdateOptions,
   OptimizeOptions,
   Version,
   WriteProgress,
-  FtsToken,
-  TokenizeTableOptions,
   LsmWriteSpec,
   ColumnAlteration,
   FieldMetadataUpdate,
@@ -176,79 +150,6 @@ export {
 } from "./arrow";
 export { IntoSql, packBits } from "./util";
 
-/**
- * Options for tokenizing a full-text search query without a table index.
- */
-export interface TokenizeOptions {
-  /**
-   * The tokenizer to use. The default is "simple".
-   */
-  baseTokenizer?: BaseTokenizer;
-
-  /** Language for stemming and stop words. */
-  language?: string;
-
-  /** Maximum token length; tokens longer than this are ignored. */
-  maxTokenLength?: number;
-
-  /** Whether to lowercase tokens. */
-  lowercase?: boolean;
-
-  /** Whether to stem tokens. */
-  stem?: boolean;
-
-  /** Whether to remove stop words. */
-  removeStopWords?: boolean;
-
-  /**
-   * Custom stop words that replace the built-in list for `language`.
-   *
-   * This option only affects tokenization when `removeStopWords` is true.
-   *
-   * `undefined` keeps the built-in language list. An empty array explicitly
-   * replaces it with no stop words.
-   */
-  customStopWords?: string[];
-
-  /** Whether to fold ASCII characters. */
-  asciiFolding?: boolean;
-
-  /** N-gram minimum length. */
-  ngramMinLength?: number;
-
-  /** N-gram maximum length. */
-  ngramMaxLength?: number;
-
-  /** Whether to only emit token prefixes for the n-gram tokenizer. */
-  prefixOnly?: boolean;
-}
-
-/**
- * Tokenize a full-text search query using an explicit tokenizer.
- *
- * This does not require a table or FTS index. The tokenizer options match
- * {@link Index.fts}.
- */
-export async function tokenize(
-  query: string,
-  options?: Partial<TokenizeOptions>,
-): Promise<FtsToken[]> {
-  return await nativeTokenize(
-    query,
-    options?.baseTokenizer,
-    options?.language,
-    options?.maxTokenLength,
-    options?.lowercase,
-    options?.stem,
-    options?.removeStopWords,
-    options?.customStopWords,
-    options?.asciiFolding,
-    options?.ngramMinLength,
-    options?.ngramMaxLength,
-    options?.prefixOnly,
-  );
-}
-
 /**
  * Connect to a LanceDB instance at the given URI.
  *

nodejs/lancedb/indices.ts

@@ -486,16 +486,6 @@ export interface IvfFlatOptions {
   sampleRate?: number;
 }
 
-export type BaseTokenizer =
-  | "simple"
-  | "whitespace"
-  | "raw"
-  | "ngram"
-  | "icu"
-  | "icu/split"
-  | `jieba/${string}`
-  | `lindera/${string}`;
-
 /**
  * Options to create a full text search index
  */
@@ -519,12 +509,8 @@ export interface FtsOptions {
    * "whitespace" - Whitespace tokenizer. This tokenizer splits the text into tokens using whitespace as a delimiter.
    *
    * "raw" - Raw tokenizer. This tokenizer does not split the text into tokens and indexes the entire text as a single token.
-   *
-   * "icu" - ICU dictionary-based word segmentation.
-   *
-   * "icu/split" - ICU segmentation with simple-style delimiter splitting.
    */
-  baseTokenizer?: BaseTokenizer;
+  baseTokenizer?: "simple" | "whitespace" | "raw" | "ngram";
 
   /**
    * language for stemming and stop words
@@ -553,16 +539,6 @@ export interface FtsOptions {
    */
   removeStopWords?: boolean;
 
-  /**
-   * Custom stop words that replace the built-in list for `language`.
-   *
-   * This option only affects tokenization when `removeStopWords` is true.
-   *
-   * `undefined` keeps the built-in language list. An empty array explicitly
-   * replaces it with no stop words.
-   */
-  customStopWords?: string[];
-
   /**
    * whether to remove punctuation
    */
@@ -582,14 +558,6 @@ export interface FtsOptions {
    * whether to only index the prefix of the token for ngram tokenizer
    */
   prefixOnly?: boolean;
-
-  /**
-   * Number of documents per compressed posting block.
-   *
-   * The default is 128. Supported values are 128 and 256. A value of 256 uses
-   * the experimental FTS V3 format and may introduce breaking changes.
-   */
-  blockSize?: 128 | 256;
 }
 
 export class Index {
@@ -765,12 +733,10 @@ export class Index {
         options?.lowercase,
         options?.stem,
         options?.removeStopWords,
-        options?.customStopWords,
         options?.asciiFolding,
         options?.ngramMinLength,
         options?.ngramMaxLength,
         options?.prefixOnly,
-        options?.blockSize,
       ),
     );
   }

nodejs/lancedb/merge.ts

@@ -88,17 +88,21 @@ export class MergeInsertBuilder {
     );
   }
   /**
-   * Control MemWAL routing for this merge.
+   * Controls whether the merge uses the MemWAL LSM write path.
    *
    * By default (unset), a `mergeInsert` on a table with an LSM write spec is
-   * routed through Lance's MemWAL shard writer, and a table without one uses the
-   * standard path.
+   * routed through Lance's MemWAL shard writer, and a table without one uses
+   * the standard path. Pass `false` to force the standard path even when a
+   * spec is set. Pass `true` to require a spec — `mergeInsert` rejects if none
+   * is installed.
    *
-   * @param enable - `true` forces MemWAL routing and errors if the table has no
-   * LSM write spec. `false` forces the standard write path even when a spec is set.
+   * @param useLsmWrite - Whether to use the LSM write path.
    */
-  useLsm(enable: boolean): MergeInsertBuilder {
-    return new MergeInsertBuilder(this.#native.useLsm(enable), this.#schema);
+  useLsmWrite(useLsmWrite: boolean): MergeInsertBuilder {
+    return new MergeInsertBuilder(
+      this.#native.useLsmWrite(useLsmWrite),
+      this.#schema,
+    );
   }
   /**
    * Controls how an LSM merge checks that its input targets a single shard.

nodejs/lancedb/otel.ts

@@ -1,137 +0,0 @@
-// SPDX-License-Identifier: Apache-2.0
-// SPDX-FileCopyrightText: Copyright The LanceDB Authors
-
-import {
-  type Attributes,
-  type MeterProvider,
-  type ObservableResult,
-  metrics,
-} from "@opentelemetry/api";
-
-import {
-  lancedbMetricsCatalog,
-  registerLancedbMetricsRecorder,
-  snapshotLancedbMetrics,
-} from "./native";
-
-let instrumented = false;
-
-/**
- * Register LanceDB metrics as OpenTelemetry observable instruments.
- *
- * Installs a process-global metrics recorder and creates one observable
- * instrument per LanceDB metric (currently object store request counts, bytes,
- * latency, errors, and throttles) on the given (or global) `MeterProvider`. The
- * configured `MetricReader` then collects them on its own schedule.
- *
- * Counters and gauges map directly to observable counters/gauges. Because
- * OpenTelemetry has no asynchronous histogram instrument, each histogram is
- * exported Prometheus-style as cumulative `le` bucket counts (`<name>_bucket`,
- * with an `le` attribute) plus `<name>_count` and `<name>_sum`.
- *
- * Requires `@opentelemetry/api` (a dependency) and, to actually export, an
- * OpenTelemetry SDK such as `@opentelemetry/sdk-metrics`.
- *
- * @param meterProvider The provider to register instruments on. Defaults to the
- *   global provider from `@opentelemetry/api`.
- * @returns `true` if the recorder is installed and instruments are registered.
- *   `false` if a different `metrics` recorder is already installed in this
- *   process (only one global recorder is permitted), in which case a warning is
- *   emitted and no instruments are created. Calling this more than once is safe;
- *   instruments are created only on the first successful call.
- */
-export function instrumentLanceDbMetrics(
-  meterProvider?: MeterProvider,
-): boolean {
-  if (!registerLancedbMetricsRecorder()) {
-    console.warn(
-      "Could not install the LanceDB metrics recorder: another `metrics` " +
-        "recorder is already installed in this process. LanceDB metrics will " +
-        "not be exported via OpenTelemetry.",
-    );
-    return false;
-  }
-
-  if (instrumented) {
-    return true;
-  }
-
-  const provider = meterProvider ?? metrics.getMeterProvider();
-  const meter = provider.getMeter("lancedb");
-
-  const scalarCallback = (metricName: string) => (result: ObservableResult) => {
-    for (const point of snapshotLancedbMetrics()) {
-      if (point.name === metricName && point.value != null) {
-        result.observe(point.value, point.attributes);
-      }
-    }
-  };
-
-  const bucketCallback = (metricName: string) => (result: ObservableResult) => {
-    for (const point of snapshotLancedbMetrics()) {
-      if (point.name !== metricName || point.buckets == null) {
-        continue;
-      }
-      for (const bucket of point.buckets) {
-        const attributes: Attributes = {
-          ...point.attributes,
-          le: bucket.le,
-        };
-        result.observe(bucket.cumulativeCount, attributes);
-      }
-    }
-  };
-
-  const fieldCallback =
-    (metricName: string, field: "count" | "sum") =>
-    (result: ObservableResult) => {
-      for (const point of snapshotLancedbMetrics()) {
-        if (point.name !== metricName) {
-          continue;
-        }
-        const value = point[field];
-        if (value != null) {
-          result.observe(value, point.attributes);
-        }
-      }
-    };
-
-  for (const desc of lancedbMetricsCatalog()) {
-    const unit = desc.unit ?? "";
-    if (desc.kind === "counter") {
-      const counter = meter.createObservableCounter(desc.name, {
-        unit,
-        description: desc.description,
-      });
-      counter.addCallback(scalarCallback(desc.name));
-    } else if (desc.kind === "gauge") {
-      const gauge = meter.createObservableGauge(desc.name, {
-        unit,
-        description: desc.description,
-      });
-      gauge.addCallback(scalarCallback(desc.name));
-    } else if (desc.kind === "histogram") {
-      // `_bucket` and `_count` observe cumulative sample counts, not the
-      // histogram's measured quantity, so they are unitless; only `_sum`
-      // carries the histogram's unit.
-      const bucket = meter.createObservableCounter(`${desc.name}_bucket`, {
-        description: `${desc.description} (cumulative buckets)`,
-      });
-      bucket.addCallback(bucketCallback(desc.name));
-
-      const count = meter.createObservableCounter(`${desc.name}_count`, {
-        description: `${desc.description} (count)`,
-      });
-      count.addCallback(fieldCallback(desc.name, "count"));
-
-      const sum = meter.createObservableCounter(`${desc.name}_sum`, {
-        unit,
-        description: `${desc.description} (sum)`,
-      });
-      sum.addCallback(fieldCallback(desc.name, "sum"));
-    }
-  }
-
-  instrumented = true;
-  return true;
-}

nodejs/lancedb/query.ts

@@ -79,8 +79,6 @@ export interface QueryExecutionOptions {
   timeoutMs?: number;
 }
 
-export type AnalyzePlanDistributedMetrics = "aggregate" | "per_worker" | "full";
-
 export interface ColumnOrdering {
   columnName: string;
   ascending?: boolean;
@@ -313,20 +311,13 @@ export class QueryBase<
    *        KNNVectorDistance: metric=l2, metrics=[output_rows=1, elapsed_compute=114.333µs, output_batches=1]
    *         LanceScan: uri=/path/to/data, projection=[vector], row_id=true, row_addr=false, ordered=false, metrics=[output_rows=1, elapsed_compute=103.626µs, bytes_read=549, iops=2, requests=2]
    *
-   * @param distributedMetrics - How distributed worker metrics are displayed for remote query plans.
-   * Defaults to `"aggregate"`.
    * @returns A query execution plan with runtime metrics for each step.
    */
-  async analyzePlan(
-    distributedMetrics?: AnalyzePlanDistributedMetrics,
-  ): Promise<string> {
-    const distributedMetricsMode = distributedMetrics ?? "aggregate";
+  async analyzePlan(): Promise<string> {
     if (this.inner instanceof Promise) {
-      return this.inner.then((inner) =>
-        inner.analyzePlan(distributedMetricsMode),
-      );
+      return this.inner.then((inner) => inner.analyzePlan());
     } else {
-      return this.inner.analyzePlan(distributedMetricsMode);
+      return this.inner.analyzePlan();
     }
   }
 
@@ -460,30 +451,6 @@ export class StandardQueryBase<
     this.doCall((inner: NativeQueryType) => inner.fastSearch());
     return this;
   }
-
-  /**
-   * Control MemWAL read routing for this query.
-   *
-   * By default (unset), when the table carries a MemWAL write spec (see
-   * {@link Table#setLsmWriteSpec}), reads are routed through the LSM scanner so
-   * they also return data written via the `mergeInsert` LSM path that has not yet
-   * been compacted into the base table (the active/frozen in-memory memtables and
-   * the flushed generations), deduplicated by primary key; a table without a spec
-   * reads the base table.
-   *
-   * @param enable - `true` forces the LSM scanner and errors if the table has no
-   * MemWAL write spec. `false` bypasses the MemWAL and reads the base table only,
-   * even when a spec is present.
-   *
-   * Note: the LSM scanner does not support every query shape (e.g. reranking,
-   * hybrid search, `orderBy`). On a MemWAL table those shapes error unless
-   * `useLsm(false)` is set, because a base-only read would silently exclude
-   * un-compacted MemWAL data.
-   */
-  useLsm(enable: boolean): this {
-    this.doCall((inner: NativeQueryType) => inner.useLsm(enable));
-    return this;
-  }
 }
 
 /**
@@ -772,20 +739,6 @@ export class TakeQuery extends QueryBase<NativeTakeQuery> {
   constructor(inner: NativeTakeQuery) {
     super(inner);
   }
-
-  /**
-   * Control MemWAL read routing for this take query.
-   *
-   * `false` bypasses the MemWAL and reads the base table only — the escape hatch,
-   * since take-by-row-id/offset is not supported on the LSM scanner and, on a
-   * MemWAL table, auto-routes to it and errors otherwise.
-   *
-   * @param enable - `false` reads the base table only.
-   */
-  useLsm(enable: boolean): this {
-    this.doCall((inner: NativeTakeQuery) => inner.useLsm(enable));
-    return this;
-  }
 }
 
 /** A builder for LanceDB queries.

nodejs/lancedb/sanitize.ts

@@ -9,7 +9,7 @@
 // comes from the exact same library instance.  This is not always the case
 // and so we must sanitize the input to ensure that it is compatible.
 
-import { BufferType, Data, Vector } from "apache-arrow";
+import { BufferType, Data } from "apache-arrow";
 import type { IntBitWidth, TKeys, TimeBitWidth } from "apache-arrow/type";
 import {
   Binary,
@@ -74,20 +74,6 @@ import {
   Utf8,
 } from "./arrow";
 
-type SanitizationContext = {
-  types: WeakMap<object, DataType>;
-  vectors: WeakMap<object, Vector>;
-  data: WeakMap<object, Data<DataType>>;
-};
-
-function createSanitizationContext(): SanitizationContext {
-  return {
-    types: new WeakMap(),
-    vectors: new WeakMap(),
-    data: new WeakMap(),
-  };
-}
-
 export function sanitizeMetadata(
   metadataLike?: unknown,
 ): Map<string, string> | undefined {
@@ -98,7 +84,7 @@ export function sanitizeMetadata(
     throw Error("Expected metadata, if present, to be a Map<string, string>");
   }
   for (const item of metadataLike) {
-    if (typeof item[0] !== "string" || typeof item[1] !== "string") {
+    if (!(typeof item[0] === "string" || !(typeof item[1] === "string"))) {
       throw Error(
         "Expected metadata, if present, to be a Map<string, string> but it had non-string keys or values",
       );
@@ -200,13 +186,6 @@ export function sanitizeInterval(typeLike: object) {
 }
 
 export function sanitizeList(typeLike: object) {
-  return sanitizeListWithContext(typeLike, createSanitizationContext());
-}
-
-function sanitizeListWithContext(
-  typeLike: object,
-  context: SanitizationContext,
-) {
   if (!("children" in typeLike) || !Array.isArray(typeLike.children)) {
     throw Error(
       "Expected a List type to have an array-like `children` property",
@@ -215,35 +194,19 @@ function sanitizeListWithContext(
   if (typeLike.children.length !== 1) {
     throw Error("Expected a List type to have exactly one child");
   }
-  return new List(sanitizeFieldWithContext(typeLike.children[0], context));
+  return new List(sanitizeField(typeLike.children[0]));
 }
 
 export function sanitizeStruct(typeLike: object) {
-  return sanitizeStructWithContext(typeLike, createSanitizationContext());
-}
-
-function sanitizeStructWithContext(
-  typeLike: object,
-  context: SanitizationContext,
-) {
   if (!("children" in typeLike) || !Array.isArray(typeLike.children)) {
     throw Error(
       "Expected a Struct type to have an array-like `children` property",
     );
   }
-  return new Struct(
-    typeLike.children.map((child) => sanitizeFieldWithContext(child, context)),
-  );
+  return new Struct(typeLike.children.map((child) => sanitizeField(child)));
 }
 
 export function sanitizeUnion(typeLike: object) {
-  return sanitizeUnionWithContext(typeLike, createSanitizationContext());
-}
-
-function sanitizeUnionWithContext(
-  typeLike: object,
-  context: SanitizationContext,
-) {
   if (
     !("typeIds" in typeLike) ||
     !("mode" in typeLike) ||
@@ -263,7 +226,7 @@ function sanitizeUnionWithContext(
     typeLike.mode,
     // biome-ignore lint/suspicious/noExplicitAny: skip
     typeLike.typeIds as any,
-    typeLike.children.map((child) => sanitizeFieldWithContext(child, context)),
+    typeLike.children.map((child) => sanitizeField(child)),
   );
 }
 
@@ -271,19 +234,6 @@ export function sanitizeTypedUnion(
   typeLike: object,
   // eslint-disable-next-line @typescript-eslint/naming-convention
   UnionType: typeof DenseUnion | typeof SparseUnion,
-) {
-  return sanitizeTypedUnionWithContext(
-    typeLike,
-    UnionType,
-    createSanitizationContext(),
-  );
-}
-
-function sanitizeTypedUnionWithContext(
-  typeLike: object,
-  // eslint-disable-next-line @typescript-eslint/naming-convention
-  UnionType: typeof DenseUnion | typeof SparseUnion,
-  context: SanitizationContext,
 ) {
   if (!("typeIds" in typeLike)) {
     throw Error(
@@ -298,7 +248,7 @@ function sanitizeTypedUnionWithContext(
 
   return new UnionType(
     typeLike.typeIds as Int32Array | number[],
-    typeLike.children.map((child) => sanitizeFieldWithContext(child, context)),
+    typeLike.children.map((child) => sanitizeField(child)),
   );
 }
 
@@ -312,16 +262,6 @@ export function sanitizeFixedSizeBinary(typeLike: object) {
 }
 
 export function sanitizeFixedSizeList(typeLike: object) {
-  return sanitizeFixedSizeListWithContext(
-    typeLike,
-    createSanitizationContext(),
-  );
-}
-
-function sanitizeFixedSizeListWithContext(
-  typeLike: object,
-  context: SanitizationContext,
-) {
   if (!("listSize" in typeLike) || typeof typeLike.listSize !== "number") {
     throw Error("Expected a FixedSizeList type to have a `listSize` property");
   }
@@ -335,18 +275,11 @@ function sanitizeFixedSizeListWithContext(
   }
   return new FixedSizeList(
     typeLike.listSize,
-    sanitizeFieldWithContext(typeLike.children[0], context),
+    sanitizeField(typeLike.children[0]),
   );
 }
 
 export function sanitizeMap(typeLike: object) {
-  return sanitizeMapWithContext(typeLike, createSanitizationContext());
-}
-
-function sanitizeMapWithContext(
-  typeLike: object,
-  context: SanitizationContext,
-) {
   if (!("children" in typeLike) || !Array.isArray(typeLike.children)) {
     throw Error(
       "Expected a Map type to have an array-like `children` property",
@@ -355,12 +288,10 @@ function sanitizeMapWithContext(
   if (!("keysSorted" in typeLike) || typeof typeLike.keysSorted !== "boolean") {
     throw Error("Expected a Map type to have a `keysSorted` property");
   }
-  if (typeLike.children.length !== 1) {
-    throw Error("Expected a Map type to have exactly one child");
-  }
 
   return new Map_(
-    sanitizeFieldWithContext(typeLike.children[0], context),
+    // biome-ignore lint/suspicious/noExplicitAny: skip
+    typeLike.children.map((field) => sanitizeField(field)) as any,
     typeLike.keysSorted,
   );
 }
@@ -373,13 +304,6 @@ export function sanitizeDuration(typeLike: object) {
 }
 
 export function sanitizeDictionary(typeLike: object) {
-  return sanitizeDictionaryWithContext(typeLike, createSanitizationContext());
-}
-
-function sanitizeDictionaryWithContext(
-  typeLike: object,
-  context: SanitizationContext,
-) {
   if (!("id" in typeLike) || typeof typeLike.id !== "number") {
     throw Error("Expected a Dictionary type to have an `id` property");
   }
@@ -393,8 +317,8 @@ function sanitizeDictionaryWithContext(
     throw Error("Expected a Dictionary type to have an `isOrdered` property");
   }
   return new Dictionary(
-    sanitizeTypeWithContext(typeLike.dictionary, context),
-    sanitizeTypeWithContext(typeLike.indices, context) as TKeys,
+    sanitizeType(typeLike.dictionary),
+    sanitizeType(typeLike.indices) as TKeys,
     typeLike.id,
     typeLike.isOrdered,
   );
@@ -402,23 +326,12 @@ function sanitizeDictionaryWithContext(
 
 // biome-ignore lint/suspicious/noExplicitAny: skip
 export function sanitizeType(typeLike: unknown): DataType<any> {
-  return sanitizeTypeWithContext(typeLike, createSanitizationContext());
-}
-
-function sanitizeTypeWithContext(
-  typeLike: unknown,
-  context: SanitizationContext,
-): DataType {
   if (typeof typeLike === "string") {
     return dataTypeFromName(typeLike);
   }
   if (typeof typeLike !== "object" || typeLike === null) {
     throw Error("Expected a Type but object was null/undefined");
   }
-  const cached = context.types.get(typeLike);
-  if (cached !== undefined) {
-    return cached;
-  }
   if (
     !("typeId" in typeLike) ||
     !(
@@ -437,16 +350,6 @@ function sanitizeTypeWithContext(
     throw Error("Type's typeId property was not a function or number");
   }
 
-  const type = sanitizeTypeById(typeLike, typeId, context);
-  context.types.set(typeLike, type);
-  return type;
-}
-
-function sanitizeTypeById(
-  typeLike: object,
-  typeId: Type,
-  context: SanitizationContext,
-): DataType {
   switch (typeId) {
     case Type.NONE:
       throw Error("Received a Type with a typeId of NONE");
@@ -473,21 +376,21 @@ function sanitizeTypeById(
     case Type.Interval:
       return sanitizeInterval(typeLike);
     case Type.List:
-      return sanitizeListWithContext(typeLike, context);
+      return sanitizeList(typeLike);
     case Type.Struct:
-      return sanitizeStructWithContext(typeLike, context);
+      return sanitizeStruct(typeLike);
     case Type.Union:
-      return sanitizeUnionWithContext(typeLike, context);
+      return sanitizeUnion(typeLike);
     case Type.FixedSizeBinary:
       return sanitizeFixedSizeBinary(typeLike);
     case Type.FixedSizeList:
-      return sanitizeFixedSizeListWithContext(typeLike, context);
+      return sanitizeFixedSizeList(typeLike);
     case Type.Map:
-      return sanitizeMapWithContext(typeLike, context);
+      return sanitizeMap(typeLike);
     case Type.Duration:
       return sanitizeDuration(typeLike);
     case Type.Dictionary:
-      return sanitizeDictionaryWithContext(typeLike, context);
+      return sanitizeDictionary(typeLike);
     case Type.Int8:
       return new Int8();
     case Type.Int16:
@@ -531,9 +434,9 @@ function sanitizeTypeById(
     case Type.TimestampSecond:
       return sanitizeTypedTimestamp(typeLike, TimestampSecond);
     case Type.DenseUnion:
-      return sanitizeTypedUnionWithContext(typeLike, DenseUnion, context);
+      return sanitizeTypedUnion(typeLike, DenseUnion);
     case Type.SparseUnion:
-      return sanitizeTypedUnionWithContext(typeLike, SparseUnion, context);
+      return sanitizeTypedUnion(typeLike, SparseUnion);
     case Type.IntervalDayTime:
       return new IntervalDayTime();
     case Type.IntervalYearMonth:
@@ -552,13 +455,6 @@ function sanitizeTypeById(
 }
 
 export function sanitizeField(fieldLike: unknown): Field {
-  return sanitizeFieldWithContext(fieldLike, createSanitizationContext());
-}
-
-function sanitizeFieldWithContext(
-  fieldLike: unknown,
-  context: SanitizationContext,
-): Field {
   if (fieldLike instanceof Field) {
     return fieldLike;
   }
@@ -576,7 +472,7 @@ function sanitizeFieldWithContext(
   }
   let type: DataType;
   try {
-    type = sanitizeTypeWithContext(fieldLike.type, context);
+    type = sanitizeType(fieldLike.type);
   } catch (error: unknown) {
     throw Error(
       `Unable to sanitize type for field: ${fieldLike.name} due to error: ${error}`,
@@ -606,13 +502,6 @@ function sanitizeFieldWithContext(
  * than lancedb is using.
  */
 export function sanitizeSchema(schemaLike: SchemaLike): Schema {
-  return sanitizeSchemaWithContext(schemaLike, createSanitizationContext());
-}
-
-function sanitizeSchemaWithContext(
-  schemaLike: SchemaLike,
-  context: SanitizationContext,
-): Schema {
   if (schemaLike instanceof Schema) {
     return schemaLike;
   }
@@ -634,7 +523,7 @@ function sanitizeSchemaWithContext(
     );
   }
   const sanitizedFields = schemaLike.fields.map((field) =>
-    sanitizeFieldWithContext(field, context),
+    sanitizeField(field),
   );
   return new Schema(sanitizedFields, metadata);
 }
@@ -656,18 +545,13 @@ export function sanitizeTable(tableLike: TableLike): Table {
       "The table passed in does not appear to be a table (no 'columns' property)",
     );
   }
-  const context = createSanitizationContext();
-  const schema = sanitizeSchemaWithContext(tableLike.schema, context);
-  const batches = tableLike.batches.map((batch) =>
-    sanitizeRecordBatch(batch, context),
-  );
+  const schema = sanitizeSchema(tableLike.schema);
+
+  const batches = tableLike.batches.map(sanitizeRecordBatch);
   return new Table(schema, batches);
 }
 
-function sanitizeRecordBatch(
-  batchLike: RecordBatchLike,
-  context: SanitizationContext,
-): RecordBatch {
+function sanitizeRecordBatch(batchLike: RecordBatchLike): RecordBatch {
   if (batchLike instanceof RecordBatch) {
     return batchLike;
   }
@@ -684,43 +568,19 @@ function sanitizeRecordBatch(
       "The record batch passed in does not appear to be a record batch (no 'data' property)",
     );
   }
-  const schema = sanitizeSchemaWithContext(batchLike.schema, context);
-  const data = sanitizeData(batchLike.data, context) as Data<Struct>;
+  const schema = sanitizeSchema(batchLike.schema);
+  const data = sanitizeData(batchLike.data);
   return new RecordBatch(schema, data);
 }
-
-type DictionaryVectorLike = {
-  data: readonly DataLike[];
-};
-
-type DictionaryDataLike = DataLike & {
-  dictionary?: DictionaryVectorLike;
-};
-
 function sanitizeData(
   dataLike: DataLike,
-  context: SanitizationContext,
-): Data<DataType> {
+  // biome-ignore lint/suspicious/noExplicitAny: <explanation>
+): import("apache-arrow").Data<Struct<any>> {
   if (dataLike instanceof Data) {
     return dataLike;
   }
-  const cachedData = context.data.get(dataLike);
-  if (cachedData !== undefined) {
-    return cachedData;
-  }
-  const dictionaryLike = (dataLike as DictionaryDataLike).dictionary;
-  let dictionary: Vector | undefined;
-  if (dictionaryLike !== undefined) {
-    dictionary = context.vectors.get(dictionaryLike);
-    if (dictionary === undefined) {
-      dictionary = new Vector(
-        dictionaryLike.data.map((data) => sanitizeData(data, context)),
-      );
-      context.vectors.set(dictionaryLike, dictionary);
-    }
-  }
-  const data = new Data(
-    sanitizeTypeWithContext(dataLike.type, context),
+  return new Data(
+    dataLike.type,
     dataLike.offset,
     dataLike.length,
     dataLike.nullCount,
@@ -730,11 +590,7 @@ function sanitizeData(
       [BufferType.VALIDITY]: dataLike.nullBitmap,
       [BufferType.TYPE]: dataLike.typeIds,
     },
-    dataLike.children.map((child) => sanitizeData(child, context)),
-    dictionary,
   );
-  context.data.set(dataLike, data);
-  return data;
 }
 
 const constructorsByTypeName = {

nodejs/lancedb/table.ts

@@ -30,7 +30,6 @@ import {
   DropColumnsResult,
   IndexConfig,
   IndexStatistics,
-  Job,
   Branches as NativeBranches,
   OptimizeStats,
   TableStatistics,
@@ -159,26 +158,6 @@ export interface Version {
   metadata: Record<string, string>;
 }
 
-/** Token produced by the tokenizer configured on a full-text search index. */
-export interface FtsToken {
-  /** Token text after tokenizer filters have been applied. */
-  text: string;
-  /** Token position used by full-text query matching. */
-  position: number;
-}
-
-export type TokenizeTableOptions =
-  | {
-      /** FTS-indexed column whose tokenizer should be used. */
-      column: string;
-      indexName?: never;
-    }
-  | {
-      /** Name of the FTS index whose tokenizer should be used. */
-      indexName: string;
-      column?: never;
-    };
-
 /**
  * Specification selecting Lance's MemWAL LSM-style write path for
  * `mergeInsert`.
@@ -359,17 +338,6 @@ export abstract class Table {
     options?: Partial<IndexOptions>,
   ): Promise<void>;
 
-  /**
-   * Create an index, returning a handle to the indexing job.
-   *
-   * The job may already be complete when returned; callers must not assume
-   * the index exists until {@link Job.wait} resolves.
-   */
-  abstract createIndexAsync(
-    column: string,
-    options?: Partial<IndexOptions>,
-  ): Promise<Job>;
-
   /**
    * Drop an index from the table.
    *
@@ -617,17 +585,6 @@ export abstract class Table {
    * @returns {Promise<void>}
    */
   abstract unsetLsmWriteSpec(): Promise<void>;
-  /**
-   * Read the {@link LsmWriteSpec} currently installed on this table.
-   *
-   * Resolves to `undefined` when the MemWAL LSM write path is not enabled (no
-   * spec has been set, or it was removed with {@link Table#unsetLsmWriteSpec}).
-   * The returned spec — including its `maintainedIndexes` and
-   * `writerConfigDefaults` — mirrors what was passed to
-   * {@link Table#setLsmWriteSpec}.
-   * @returns {Promise<LsmWriteSpec | undefined>}
-   */
-  abstract getLsmWriteSpec(): Promise<LsmWriteSpec | undefined>;
   /**
    * Drain and close any cached MemWAL shard writers held for this table.
    *
@@ -748,19 +705,6 @@ export abstract class Table {
   abstract optimize(options?: Partial<OptimizeOptions>): Promise<OptimizeStats>;
   /** List all indices that have been created with {@link Table.createIndex} */
   abstract listIndices(): Promise<IndexConfig[]>;
-  /**
-   * Tokenize a full-text search query using the tokenizer configured on an FTS index.
-   *
-   * Specify exactly one of `column` or `indexName`.
-   *
-   * Model-backed tokenizers such as `jieba/*` and `lindera/*` are rebuilt in
-   * the client process from index metadata. For remote tables, this means the
-   * same tokenizer model files must also exist locally.
-   */
-  abstract tokenize(
-    query: string,
-    options: TokenizeTableOptions,
-  ): Promise<FtsToken[]>;
   /** Return the table as an arrow table */
   abstract toArrow(): Promise<ArrowTable>;
 
@@ -952,22 +896,6 @@ export class LocalTable extends Table {
     );
   }
 
-  async createIndexAsync(
-    column: string,
-    options?: Partial<IndexOptions>,
-  ): Promise<Job> {
-    // biome-ignore lint/suspicious/noExplicitAny: skip
-    const nativeIndex = (options?.config as any)?.inner;
-    return await this.inner.createIndexAsync(
-      nativeIndex,
-      column,
-      options?.replace,
-      options?.waitTimeoutSeconds,
-      options?.name,
-      options?.train,
-    );
-  }
-
   async dropIndex(name: string): Promise<void> {
     await this.inner.dropIndex(name);
   }
@@ -1163,15 +1091,6 @@ export class LocalTable extends Table {
     return await this.inner.unsetLsmWriteSpec();
   }
 
-  async getLsmWriteSpec(): Promise<LsmWriteSpec | undefined> {
-    // The native binding types `specType` as a plain `string`; narrow it back
-    // to the public union. The Rust `From` impl only ever emits one of the
-    // three valid values, so the cast is safe.
-    return ((await this.inner.getLsmWriteSpec()) ?? undefined) as
-      | LsmWriteSpec
-      | undefined;
-  }
-
   async closeLsmWriters(): Promise<void> {
     return await this.inner.closeLsmWriters();
   }
@@ -1234,17 +1153,6 @@ export class LocalTable extends Table {
     return await this.inner.listIndices();
   }
 
-  async tokenize(
-    query: string,
-    options: TokenizeTableOptions,
-  ): Promise<FtsToken[]> {
-    return await this.inner.tokenize(
-      query,
-      options?.column,
-      options?.indexName,
-    );
-  }
-
   async toArrow(): Promise<ArrowTable> {
     return await this.query().toArrow();
   }
@@ -1357,76 +1265,6 @@ export interface FieldMetadataUpdate {
   replace?: boolean;
 }
 
-/** Summary of a column in a branch diff. */
-export interface BranchColumnSummary {
-  name: string;
-  dataType: string;
-  nullable: boolean;
-}
-
-/** A column whose definition differs between main and the branch. */
-export interface BranchColumnChange {
-  name: string;
-  main: BranchColumnSummary;
-  branch: BranchColumnSummary;
-}
-
-/** Summary of an index in a branch diff. */
-export interface BranchIndexSummary {
-  indexName: string;
-  columns: string[];
-  indexType?: string;
-  status: string;
-}
-
-/** Row-level comparison between main and the branch. */
-export interface BranchRowCountSummary {
-  unchanged: number;
-  newOnBase: number;
-  newOnBranch: number;
-  staleRecompute: number;
-  inputsChanged: number;
-  deltaAvailable: boolean;
-}
-
-/** A reason why a branch cannot currently be merged. */
-export interface MergeBlocker {
-  code: string;
-  message: string;
-}
-
-/** Read-only comparison of a branch against main. */
-export interface BranchDiff {
-  fromBranch: string;
-  parentVersion: number;
-  mainVersion: number;
-  branchVersion: number;
-  baseMoved: boolean;
-  rowCountMain: number;
-  rowCountBranch: number;
-  rowSummary: BranchRowCountSummary;
-  addedColumns: BranchColumnSummary[];
-  removedColumns: BranchColumnSummary[];
-  changedColumns: BranchColumnChange[];
-  addedIndexes: BranchIndexSummary[];
-  removedIndexes: BranchIndexSummary[];
-  mergeable: boolean;
-  mergeBlockers: MergeBlocker[];
-}
-
-/** Changes that would be, or were, promoted by a branch merge. */
-export interface MergePreview {
-  promotedColumns: string[];
-}
-
-/** Result of previewing or attempting a branch merge. */
-export interface MergeBranchResult {
-  status: "ready" | "rejected" | "notImplemented" | "merged" | "unknown";
-  diff: BranchDiff;
-  preview: MergePreview;
-  mainVersionAfter?: number;
-}
-
 /**
  * Branch manager for a {@link Table}.
  *
@@ -1479,28 +1317,4 @@ export class Branches {
   async delete(name: string): Promise<void> {
     return await this.#inner.delete(name);
   }
-
-  /** Compare a branch against main without modifying either branch. */
-  async diff(fromBranch: string): Promise<BranchDiff> {
-    return (await this.#inner.diff(fromBranch)) as unknown as BranchDiff;
-  }
-
-  /**
-   * Merge a branch into main.
-   *
-   * Set `dryRun` to `true` to preview the merge. A rejected merge resolves
-   * with `status: "rejected"` instead of throwing.
-   *
-   * @param fromBranch Branch to merge from.
-   * @param dryRun When true, only preview the merge. Defaults to false.
-   */
-  async merge(
-    fromBranch: string,
-    dryRun: boolean = false,
-  ): Promise<MergeBranchResult> {
-    return (await this.#inner.merge(
-      fromBranch,
-      dryRun,
-    )) as unknown as MergeBranchResult;
-  }
 }

nodejs/npm/darwin-arm64/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@lancedb/lancedb-darwin-arm64",
-	"version": "0.37.1",
+	"version": "0.31.0-beta.6",
 	"os": ["darwin"],
 	"cpu": ["arm64"],
 	"main": "lancedb.darwin-arm64.node",

nodejs/npm/linux-arm64-gnu/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@lancedb/lancedb-linux-arm64-gnu",
-	"version": "0.37.1",
+	"version": "0.31.0-beta.6",
 	"os": ["linux"],
 	"cpu": ["arm64"],
 	"main": "lancedb.linux-arm64-gnu.node",

nodejs/npm/linux-arm64-musl/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@lancedb/lancedb-linux-arm64-musl",
-	"version": "0.37.1",
+	"version": "0.31.0-beta.6",
 	"os": ["linux"],
 	"cpu": ["arm64"],
 	"main": "lancedb.linux-arm64-musl.node",

nodejs/npm/linux-x64-gnu/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@lancedb/lancedb-linux-x64-gnu",
-	"version": "0.37.1",
+	"version": "0.31.0-beta.6",
 	"os": ["linux"],
 	"cpu": ["x64"],
 	"main": "lancedb.linux-x64-gnu.node",

nodejs/npm/linux-x64-musl/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@lancedb/lancedb-linux-x64-musl",
-	"version": "0.37.1",
+	"version": "0.31.0-beta.6",
 	"os": ["linux"],
 	"cpu": ["x64"],
 	"main": "lancedb.linux-x64-musl.node",

nodejs/npm/win32-arm64-msvc/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lancedb/lancedb-win32-arm64-msvc",
-  "version": "0.37.1",
+  "version": "0.31.0-beta.6",
   "os": [
     "win32"
   ],

nodejs/npm/win32-x64-msvc/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@lancedb/lancedb-win32-x64-msvc",
-	"version": "0.37.1",
+	"version": "0.31.0-beta.6",
 	"os": ["win32"],
 	"cpu": ["x64"],
 	"main": "lancedb.win32-x64-msvc.node",

nodejs/package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "@lancedb/lancedb",
-  "version": "0.37.1",
+  "version": "0.31.0-beta.6",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "@lancedb/lancedb",
-      "version": "0.37.1",
+      "version": "0.31.0-beta.6",
       "cpu": [
         "x64",
         "arm64"
@@ -18,7 +18,6 @@
         "win32"
       ],
       "dependencies": {
-        "@opentelemetry/api": "^1.9.0",
         "reflect-metadata": "^0.2.2"
       },
       "devDependencies": {
@@ -28,7 +27,6 @@
         "@biomejs/biome": "^1.7.3",
         "@jest/globals": "^29.7.0",
         "@napi-rs/cli": "3.7.0",
-        "@opentelemetry/sdk-metrics": "^1.30.0",
         "@types/axios": "^0.14.0",
         "@types/jest": "^29.1.2",
         "@types/node": "22.7.4",
@@ -55,13 +53,7 @@
         "openai": "4.29.2"
       },
       "peerDependencies": {
-        "@types/node": ">=18",
         "apache-arrow": ">=15.0.0 <=18.1.0"
-      },
-      "peerDependenciesMeta": {
-        "@types/node": {
-          "optional": true
-        }
       }
     },
     "node_modules/@aws-crypto/crc32": {
@@ -4156,75 +4148,6 @@
         "@octokit/openapi-types": "^27.0.0"
       }
     },
-    "node_modules/@opentelemetry/api": {
-      "version": "1.9.1",
-      "resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.1.tgz",
-      "integrity": "sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==",
-      "license": "Apache-2.0",
-      "engines": {
-        "node": ">=8.0.0"
-      }
-    },
-    "node_modules/@opentelemetry/core": {
-      "version": "1.30.1",
-      "resolved": "https://registry.npmjs.org/@opentelemetry/core/-/core-1.30.1.tgz",
-      "integrity": "sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==",
-      "dev": true,
-      "license": "Apache-2.0",
-      "dependencies": {
-        "@opentelemetry/semantic-conventions": "1.28.0"
-      },
-      "engines": {
-        "node": ">=14"
-      },
-      "peerDependencies": {
-        "@opentelemetry/api": ">=1.0.0 <1.10.0"
-      }
-    },
-    "node_modules/@opentelemetry/resources": {
-      "version": "1.30.1",
-      "resolved": "https://registry.npmjs.org/@opentelemetry/resources/-/resources-1.30.1.tgz",
-      "integrity": "sha512-5UxZqiAgLYGFjS4s9qm5mBVo433u+dSPUFWVWXmLAD4wB65oMCoXaJP1KJa9DIYYMeHu3z4BZcStG3LC593cWA==",
-      "dev": true,
-      "license": "Apache-2.0",
-      "dependencies": {
-        "@opentelemetry/core": "1.30.1",
-        "@opentelemetry/semantic-conventions": "1.28.0"
-      },
-      "engines": {
-        "node": ">=14"
-      },
-      "peerDependencies": {
-        "@opentelemetry/api": ">=1.0.0 <1.10.0"
-      }
-    },
-    "node_modules/@opentelemetry/sdk-metrics": {
-      "version": "1.30.1",
-      "resolved": "https://registry.npmjs.org/@opentelemetry/sdk-metrics/-/sdk-metrics-1.30.1.tgz",
-      "integrity": "sha512-q9zcZ0Okl8jRgmy7eNW3Ku1XSgg3sDLa5evHZpCwjspw7E8Is4K/haRPDJrBcX3YSn/Y7gUvFnByNYEKQNbNog==",
-      "dev": true,
-      "license": "Apache-2.0",
-      "dependencies": {
-        "@opentelemetry/core": "1.30.1",
-        "@opentelemetry/resources": "1.30.1"
-      },
-      "engines": {
-        "node": ">=14"
-      },
-      "peerDependencies": {
-        "@opentelemetry/api": ">=1.3.0 <1.10.0"
-      }
-    },
-    "node_modules/@opentelemetry/semantic-conventions": {
-      "version": "1.28.0",
-      "resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.28.0.tgz",
-      "integrity": "sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==",
-      "dev": true,
-      "license": "Apache-2.0",
-      "engines": {
-        "node": ">=14"
-      }
-    },
     "node_modules/@protobufjs/aspromise": {
       "version": "1.1.2",
       "resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz",

nodejs/package.json

@@ -11,7 +11,7 @@
     "ann"
   ],
   "private": false,
-  "version": "0.37.1",
+  "version": "0.31.0-beta.6",
   "main": "dist/index.js",
   "exports": {
     ".": "./dist/index.js",
@@ -44,7 +44,6 @@
     "@biomejs/biome": "^1.7.3",
     "@jest/globals": "^29.7.0",
     "@napi-rs/cli": "3.7.0",
-    "@opentelemetry/sdk-metrics": "^1.30.0",
     "@types/axios": "^0.14.0",
     "@types/jest": "^29.1.2",
     "@types/node": "22.7.4",
@@ -93,7 +92,6 @@
     "version": "napi version"
   },
   "dependencies": {
-    "@opentelemetry/api": "^1.9.0",
     "reflect-metadata": "^0.2.2"
   },
   "optionalDependencies": {
@@ -101,12 +99,6 @@
     "openai": "4.29.2"
   },
   "peerDependencies": {
-    "@types/node": ">=18",
     "apache-arrow": ">=15.0.0 <=18.1.0"
-  },
-  "peerDependenciesMeta": {
-    "@types/node": {
-      "optional": true
-    }
   }
 }

nodejs/pnpm-lock.yaml

@@ -8,9 +8,6 @@ importers:
 
   .:
     dependencies:
-      '@opentelemetry/api':
-        specifier: ^1.9.0
-        version: 1.9.1
       apache-arrow:
         specifier: '>=15.0.0 <=18.1.0'
         version: 18.1.0
@@ -36,9 +33,6 @@ importers:
       '@napi-rs/cli':
         specifier: 3.7.0
         version: 3.7.0(@emnapi/core@1.10.0)(@emnapi/runtime@1.10.0)(@types/node@22.7.4)
-      '@opentelemetry/sdk-metrics':
-        specifier: ^1.30.0
-        version: 1.30.1(@opentelemetry/api@1.9.1)
       '@types/axios':
         specifier: ^0.14.0
         version: 0.14.4
@@ -1313,32 +1307,6 @@ packages:
   '@octokit/types@16.0.0':
     resolution: {integrity: sha512-sKq+9r1Mm4efXW1FCk7hFSeJo4QKreL/tTbR0rz/qx/r1Oa2VV83LTA/H/MuCOX7uCIJmQVRKBcbmWoySjAnSg==}
 
-  '@opentelemetry/api@1.9.1':
-    resolution: {integrity: sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==}
-    engines: {node: '>=8.0.0'}
-
-  '@opentelemetry/core@1.30.1':
-    resolution: {integrity: sha512-OOCM2C/QIURhJMuKaekP3TRBxBKxG/TWWA0TL2J6nXUtDnuCtccy49LUJF8xPFXMX+0LMcxFpCo8M9cGY1W6rQ==}
-    engines: {node: '>=14'}
-    peerDependencies:
-      '@opentelemetry/api': '>=1.0.0 <1.10.0'
-
-  '@opentelemetry/resources@1.30.1':
-    resolution: {integrity: sha512-5UxZqiAgLYGFjS4s9qm5mBVo433u+dSPUFWVWXmLAD4wB65oMCoXaJP1KJa9DIYYMeHu3z4BZcStG3LC593cWA==}
-    engines: {node: '>=14'}
-    peerDependencies:
-      '@opentelemetry/api': '>=1.0.0 <1.10.0'
-
-  '@opentelemetry/sdk-metrics@1.30.1':
-    resolution: {integrity: sha512-q9zcZ0Okl8jRgmy7eNW3Ku1XSgg3sDLa5evHZpCwjspw7E8Is4K/haRPDJrBcX3YSn/Y7gUvFnByNYEKQNbNog==}
-    engines: {node: '>=14'}
-    peerDependencies:
-      '@opentelemetry/api': '>=1.3.0 <1.10.0'
-
-  '@opentelemetry/semantic-conventions@1.28.0':
-    resolution: {integrity: sha512-lp4qAiMTD4sNWW4DbKLBkfiMZ4jbAboJIGOQr5DvciMRI494OapieI9qiODpOt0XBr1LjIDy1xAGAnVs5supTA==}
-    engines: {node: '>=14'}
-
   '@protobufjs/aspromise@1.1.2':
     resolution: {integrity: sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==}
 
@@ -4957,27 +4925,6 @@ snapshots:
     dependencies:
       '@octokit/openapi-types': 27.0.0
 
-  '@opentelemetry/api@1.9.1': {}
-
-  '@opentelemetry/core@1.30.1(@opentelemetry/api@1.9.1)':
-    dependencies:
-      '@opentelemetry/api': 1.9.1
-      '@opentelemetry/semantic-conventions': 1.28.0
-
-  '@opentelemetry/resources@1.30.1(@opentelemetry/api@1.9.1)':
-    dependencies:
-      '@opentelemetry/api': 1.9.1
-      '@opentelemetry/core': 1.30.1(@opentelemetry/api@1.9.1)
-      '@opentelemetry/semantic-conventions': 1.28.0
-
-  '@opentelemetry/sdk-metrics@1.30.1(@opentelemetry/api@1.9.1)':
-    dependencies:
-      '@opentelemetry/api': 1.9.1
-      '@opentelemetry/core': 1.30.1(@opentelemetry/api@1.9.1)
-      '@opentelemetry/resources': 1.30.1(@opentelemetry/api@1.9.1)
-
-  '@opentelemetry/semantic-conventions@1.28.0': {}
-
   '@protobufjs/aspromise@1.1.2':
     optional: true
 

nodejs/src/connection.rs

@@ -340,69 +340,6 @@ impl Connection {
         self.get_inner()?.drop_all_tables(&ns).await.default_error()
     }
 
-    /// A `Job` handle for a server-side job by id.
-    ///
-    /// The handle is constructed without a server round trip; an unknown id
-    /// surfaces when the handle is used.
-    #[napi]
-    pub fn job(&self, job_id: String) -> napi::Result<crate::job::Job> {
-        let job = self.get_inner()?.job(job_id).default_error()?;
-        Ok(crate::job::Job::new(job))
-    }
-
-    /// List server-side jobs across the database's tables.
-    #[napi(catch_unwind)]
-    pub async fn list_jobs(&self) -> napi::Result<Vec<crate::job::JobInfo>> {
-        let jobs = self.get_inner()?.list_jobs().await.default_error()?;
-        Ok(jobs.into_iter().map(Into::into).collect())
-    }
-
-    /// Describe a single server-side job by id. `null` when the server has
-    /// no such job.
-    #[napi(catch_unwind)]
-    pub async fn get_job(
-        &self,
-        job_id: String,
-    ) -> napi::Result<Option<crate::job::JobDescription>> {
-        let description = self.get_inner()?.get_job(&job_id).await.default_error()?;
-        Ok(description.map(Into::into))
-    }
-
-    /// Request cancellation of a server-side job by id. Returns true if the
-    /// server accepted the cancellation, false if no such job exists.
-    #[napi(catch_unwind)]
-    pub async fn cancel_job(&self, job_id: String) -> napi::Result<bool> {
-        self.get_inner()?.cancel_job(&job_id).await.default_error()
-    }
-
-    /// The lifecycle event history of a server-side job (all jobs when
-    /// `job_id` is null), as an Arrow IPC stream buffer. Empty when there is
-    /// no history.
-    #[napi(catch_unwind)]
-    pub async fn job_history(&self, job_id: Option<String>) -> napi::Result<Buffer> {
-        let batches = self
-            .get_inner()?
-            .job_history(job_id.as_deref())
-            .await
-            .default_error()?;
-        let Some(first) = batches.first() else {
-            return Ok(Buffer::from(Vec::<u8>::new()));
-        };
-        let mut out = Vec::new();
-        let mut writer = arrow_ipc::writer::StreamWriter::try_new(&mut out, &first.schema())
-            .map_err(|e| napi::Error::from_reason(e.to_string()))?;
-        for batch in &batches {
-            writer
-                .write(batch)
-                .map_err(|e| napi::Error::from_reason(e.to_string()))?;
-        }
-        writer
-            .finish()
-            .map_err(|e| napi::Error::from_reason(e.to_string()))?;
-        drop(writer);
-        Ok(Buffer::from(out))
-    }
-
     #[napi(catch_unwind)]
     /// Describe a namespace and return its properties.
     pub async fn describe_namespace(

nodejs/src/index.rs

@@ -9,11 +9,8 @@ use lancedb::index::vector::{
     IvfFlatIndexBuilder, IvfHnswPqIndexBuilder, IvfHnswSqIndexBuilder, IvfPqIndexBuilder,
     IvfRqIndexBuilder,
 };
-use lancedb::tokenize as lancedb_tokenize;
 use napi_derive::napi;
 
-use crate::error::NapiErrorExt;
-use crate::table::FtsToken;
 use crate::util::parse_distance_type;
 
 #[napi]
@@ -33,67 +30,6 @@ impl Index {
     }
 }
 
-#[napi(catch_unwind)]
-#[allow(dead_code, clippy::too_many_arguments)]
-pub fn tokenize(
-    query: String,
-    base_tokenizer: Option<String>,
-    language: Option<String>,
-    max_token_length: Option<u32>,
-    lower_case: Option<bool>,
-    stem: Option<bool>,
-    remove_stop_words: Option<bool>,
-    custom_stop_words: Option<Vec<String>>,
-    ascii_folding: Option<bool>,
-    ngram_min_length: Option<u32>,
-    ngram_max_length: Option<u32>,
-    prefix_only: Option<bool>,
-) -> napi::Result<Vec<FtsToken>> {
-    let mut opts = FtsIndexBuilder::default();
-    if let Some(base_tokenizer) = base_tokenizer {
-        opts = opts.base_tokenizer(base_tokenizer);
-    }
-    if let Some(language) = language {
-        opts = opts.language(&language).map_err(|_| {
-            napi::Error::from_reason(format!(
-                "LanceDB does not support the requested language: '{}'",
-                language
-            ))
-        })?;
-    }
-    if let Some(max_token_length) = max_token_length {
-        opts = opts.max_token_length(Some(max_token_length as usize));
-    }
-    if let Some(lower_case) = lower_case {
-        opts = opts.lower_case(lower_case);
-    }
-    if let Some(stem) = stem {
-        opts = opts.stem(stem);
-    }
-    if let Some(remove_stop_words) = remove_stop_words {
-        opts = opts.remove_stop_words(remove_stop_words);
-    }
-    opts = opts.custom_stop_words(custom_stop_words);
-    if let Some(ascii_folding) = ascii_folding {
-        opts = opts.ascii_folding(ascii_folding);
-    }
-    if let Some(ngram_min_length) = ngram_min_length {
-        opts = opts.ngram_min_length(ngram_min_length);
-    }
-    if let Some(ngram_max_length) = ngram_max_length {
-        opts = opts.ngram_max_length(ngram_max_length);
-    }
-    if let Some(prefix_only) = prefix_only {
-        opts = opts.ngram_prefix_only(prefix_only);
-    }
-
-    Ok(lancedb_tokenize(&query, &opts)
-        .default_error()?
-        .into_iter()
-        .map(FtsToken::from)
-        .collect())
-}
-
 #[napi]
 impl Index {
     #[napi(factory)]
@@ -224,13 +160,11 @@ impl Index {
         lower_case: Option<bool>,
         stem: Option<bool>,
         remove_stop_words: Option<bool>,
-        custom_stop_words: Option<Vec<String>>,
         ascii_folding: Option<bool>,
         ngram_min_length: Option<u32>,
         ngram_max_length: Option<u32>,
         prefix_only: Option<bool>,
-        block_size: Option<u32>,
-    ) -> napi::Result<Self> {
+    ) -> Self {
         let mut opts = FtsIndexBuilder::default();
         if let Some(with_position) = with_position {
             opts = opts.with_position(with_position);
@@ -253,7 +187,6 @@ impl Index {
         if let Some(remove_stop_words) = remove_stop_words {
             opts = opts.remove_stop_words(remove_stop_words);
         }
-        opts = opts.custom_stop_words(custom_stop_words);
         if let Some(ascii_folding) = ascii_folding {
             opts = opts.ascii_folding(ascii_folding);
         }
@@ -266,15 +199,10 @@ impl Index {
         if let Some(prefix_only) = prefix_only {
             opts = opts.ngram_prefix_only(prefix_only);
         }
-        if let Some(block_size) = block_size {
-            opts = opts
-                .block_size(block_size as usize)
-                .map_err(|err| napi::Error::from_reason(err.to_string()))?;
-        }
 
-        Ok(Self {
+        Self {
             inner: Mutex::new(Some(LanceDbIndex::FTS(opts))),
-        })
+        }
     }
 
     #[napi(factory)]

nodejs/src/job.rs

@@ -1,123 +0,0 @@
-// SPDX-License-Identifier: Apache-2.0
-// SPDX-FileCopyrightText: Copyright The LanceDB Authors
-
-use std::sync::Arc;
-
-use napi_derive::napi;
-
-use crate::error::NapiErrorExt;
-
-/// A handle to an operation that may still be running.
-#[napi]
-pub struct Job {
-    inner: Arc<lancedb::Job>,
-}
-
-impl Job {
-    pub(crate) fn new(inner: lancedb::Job) -> Self {
-        Self {
-            inner: Arc::new(inner),
-        }
-    }
-}
-
-#[napi]
-impl Job {
-    /// Identifies the operation on the server that is running it. Operations
-    /// that run in this process have no server id. The value is opaque.
-    #[napi(getter)]
-    pub fn id(&self) -> Option<String> {
-        self.inner.id().map(str::to_string)
-    }
-
-    /// The operation's current lifecycle state: "running", "finished",
-    /// "failed", or "cancelled".
-    ///
-    /// A point snapshot; unlike {@link Job.wait} it does not block or reject
-    /// on a terminal failure state. States a newer server reports that this
-    /// client version does not know pass through as-is.
-    #[napi(catch_unwind)]
-    pub async fn status(&self) -> napi::Result<String> {
-        self.inner.status().await.default_error()
-    }
-
-    /// Wait until the operation reaches a terminal state.
-    #[napi(catch_unwind)]
-    pub async fn wait(&self) -> napi::Result<()> {
-        self.inner.wait().await.default_error()
-    }
-
-    /// Request cancellation. Cancelling a finished operation is a no-op.
-    #[napi(catch_unwind)]
-    pub async fn cancel(&self) -> napi::Result<()> {
-        self.inner.cancel().await.default_error()
-    }
-}
-
-/// A row from `Connection.listJobs`: one server-side job.
-#[napi(object)]
-pub struct JobInfo {
-    /// The job id -- what `Connection.getJob` and `Connection.cancelJob`
-    /// accept.
-    pub job_id: String,
-    /// The table the job runs against, without URI or namespace.
-    pub table: String,
-    pub job_type: String,
-    /// Lifecycle state: "running", "finished", "failed", or "cancelled".
-    pub state: String,
-    /// When the job was created, in milliseconds since the epoch.
-    pub created_at_millis: i64,
-}
-
-impl From<lancedb::database::JobInfo> for JobInfo {
-    fn from(info: lancedb::database::JobInfo) -> Self {
-        Self {
-            job_id: info.job_id,
-            table: info.table,
-            job_type: info.job_type,
-            state: info.state,
-            created_at_millis: info.created_at_millis,
-        }
-    }
-}
-
-/// The server's account of why a job failed.
-#[napi(object)]
-pub struct JobFailureInfo {
-    pub phase: Option<String>,
-    pub message: Option<String>,
-    pub retryable: Option<bool>,
-}
-
-/// A described job from `Connection.getJob`.
-#[napi(object)]
-pub struct JobDescription {
-    pub job_id: String,
-    pub job_type: String,
-    /// Lifecycle state: "running", "finished", "failed", or "cancelled".
-    pub state: String,
-    /// When the job was created, in milliseconds since the epoch.
-    pub creation_ms: i64,
-    /// The job-type-specific specification as a JSON string, when present.
-    pub spec_json: Option<String>,
-    /// Why the job failed, when the job is failed and the server reports a
-    /// reason.
-    pub failure: Option<JobFailureInfo>,
-}
-
-impl From<lancedb::database::JobDescription> for JobDescription {
-    fn from(description: lancedb::database::JobDescription) -> Self {
-        Self {
-            job_id: description.job_id,
-            job_type: description.job_type,
-            state: description.state,
-            creation_ms: description.creation_ms,
-            spec_json: (!description.spec.is_null()).then(|| description.spec.to_string()),
-            failure: description.failure.map(|failure| JobFailureInfo {
-                phase: failure.phase,
-                message: failure.message,
-                retryable: failure.retryable,
-            }),
-        }
-    }
-}

nodejs/src/lib.rs

@@ -11,9 +11,7 @@ mod error;
 mod header;
 mod index;
 mod iterator;
-mod job;
 pub mod merge;
-pub mod otel;
 pub mod permutation;
 mod query;
 pub mod remote;

nodejs/src/merge.rs

@@ -51,9 +51,9 @@ impl NativeMergeInsertBuilder {
     }
 
     #[napi]
-    pub fn use_lsm(&self, enable: bool) -> Self {
+    pub fn use_lsm_write(&self, use_lsm_write: bool) -> Self {
         let mut this = self.clone();
-        this.inner.use_lsm(enable);
+        this.inner.use_lsm_write(use_lsm_write);
         this
     }
 

nodejs/src/otel.rs

@@ -1,119 +0,0 @@
-// SPDX-License-Identifier: Apache-2.0
-// SPDX-FileCopyrightText: Copyright The LanceDB Authors
-
-//! Node.js bindings over [`lancedb::metrics_otel`].
-//!
-//! The aggregation, catalog, and histogram bucketing all live in the LanceDB
-//! core crate; this module only converts the core snapshot types into napi
-//! objects and exposes the three entry points to JavaScript, where
-//! `lancedb/otel.ts` bridges them into the user's OpenTelemetry `MeterProvider`.
-
-use std::collections::HashMap;
-
-use lancedb::metrics_otel::{MetricPoint as CoreMetricPoint, MetricValue};
-use napi_derive::napi;
-
-/// One cumulative histogram bucket: all samples with value `<= le`.
-#[napi(object)]
-pub struct MetricBucket {
-    /// The inclusive upper bound of the bucket, or `"+Inf"` for the final bucket.
-    pub le: String,
-    /// Cumulative number of samples less than or equal to `le`.
-    pub cumulative_count: f64,
-}
-
-/// One aggregated metric data point. For counters and gauges only `value` is
-/// set; for histograms `buckets` (cumulative `le` counts), `count`, and `sum`
-/// are set.
-#[napi(object)]
-pub struct MetricPoint {
-    pub name: String,
-    pub kind: String,
-    pub attributes: HashMap<String, String>,
-    pub value: Option<f64>,
-    pub buckets: Option<Vec<MetricBucket>>,
-    pub count: Option<f64>,
-    pub sum: Option<f64>,
-}
-
-impl From<CoreMetricPoint> for MetricPoint {
-    fn from(point: CoreMetricPoint) -> Self {
-        let kind = point.kind.as_str().to_string();
-        let (value, buckets, count, sum) = match point.value {
-            MetricValue::Scalar(v) => (Some(v), None, None, None),
-            MetricValue::Histogram {
-                buckets,
-                count,
-                sum,
-            } => (
-                None,
-                Some(
-                    buckets
-                        .into_iter()
-                        // Counts stay well within the f64-exact integer range
-                        // (2^53), so this cast is lossless in practice and keeps
-                        // the values plain JS numbers for OpenTelemetry.
-                        .map(|(le, cumulative_count)| MetricBucket {
-                            le,
-                            cumulative_count: cumulative_count as f64,
-                        })
-                        .collect(),
-                ),
-                Some(count as f64),
-                Some(sum),
-            ),
-        };
-        Self {
-            name: point.name,
-            kind,
-            attributes: point.attributes,
-            value,
-            buckets,
-            count,
-            sum,
-        }
-    }
-}
-
-/// A described metric, used by the JavaScript layer to create instruments up front.
-#[napi(object)]
-pub struct MetricDescription {
-    pub name: String,
-    pub kind: String,
-    pub unit: Option<String>,
-    pub description: String,
-}
-
-/// Install the LanceDB metrics recorder as the process-global `metrics` recorder.
-///
-/// Returns `true` if the recorder is installed (now or previously). Returns
-/// `false` if a *different* recorder is already installed — `metrics` allows
-/// only one global recorder per process, so LanceDB cannot coexist with another.
-#[napi]
-pub fn register_lancedb_metrics_recorder() -> bool {
-    lancedb::metrics_otel::register_metrics_recorder()
-}
-
-/// The catalog of described LanceDB metrics. Empty until the recorder is installed.
-#[napi]
-pub fn lancedb_metrics_catalog() -> Vec<MetricDescription> {
-    lancedb::metrics_otel::metrics_catalog()
-        .into_iter()
-        .map(|desc| MetricDescription {
-            name: desc.name,
-            kind: desc.kind.as_str().to_string(),
-            unit: desc.unit,
-            description: desc.description,
-        })
-        .collect()
-}
-
-/// A point-in-time snapshot of every recorded metric. Empty until the recorder
-/// is installed.
-#[napi]
-pub fn snapshot_lancedb_metrics() -> Vec<MetricPoint> {
-    lancedb::metrics_otel::snapshot_metrics()
-        .into_iter()
-        .map(MetricPoint::from)
-        .collect()
-}