refactor(lsm): remove the index-catchup activation surface

Follows the lance change that removes the feature bit. With one set of semantics there is no mode to switch into, so `require_mem_wal_index_catchup` goes from the trait, `NativeTable`, and the LSM merge module. `exclusion_watermarks` loses its `catchup_required` argument and keeps the conservative branch: an index with no `index_catchup` entry is not known to hold the compacted rows, so its generations stay readable from their SSTables. That is what makes an existing table safe to read the moment the new binary starts -- nothing is excluded until an index records that it covers it. Two tests changed because they encoded the branch that is gone, and they had conflated two different things: an index that is *caught up* and an index that is *untracked* both fell back to the compaction watermark. Untracked now retains everything, so the tests assert that and cover the genuinely-caught-up case separately. 484 lancedb lib tests pass.
feat(lsm): move catch-up activation to an explicit API
2026-09-02 19:49:00 +00:00 · 2026-08-20 18:29:56 -04:00 · 2026-08-10 09:10:05 -04:00 · 2026-08-09 23:19:21 -04:00 · 2026-08-02 00:12:47 +00:00
63 changed files with 592 additions and 4213 deletions
@@ -1,222 +0,0 @@
-name: Check doc links
-
-# Checking external links is inherently noisy: third-party sites rate-limit
-# automated clients, reject non-browser user agents, and go down temporarily.
-# Blocking pull requests on that trades a lot of false failures for very little
-# signal, so this runs on a schedule and reports findings in a single tracking
-# issue instead of failing anyone's build.
-on:
-  schedule:
-    - cron: "0 7 * * *"
-  workflow_dispatch:
-
-# The report lives in one repository-global issue, so runs must not overlap: a
-# lookup racing a create produces duplicate issues, and a healthy run closing
-# the issue while a failing run only rewrites its body would leave a broken
-# report closed. The group is deliberately ref-independent so that a manual
-# dispatch serializes against the scheduled run.
-concurrency:
-  group: docs-link-check
-  cancel-in-progress: false
-
-permissions: {}
-
-env:
-  REPORT_TITLE: "Docs link checker report"
-
-jobs:
-  scan:
-    name: Scan links
-    runs-on: ubuntu-24.04
-    # lychee-action is pinned by SHA, but its wrapper downloads the lychee
-    # release tarball at run time without verifying a digest, and hands the
-    # resulting binary a GitHub token. Release assets remain replaceable, so
-    # that binary is confined to a job whose token can only read public
-    # content; everything that writes runs in the report job below.
-    permissions:
-      contents: read
-    outputs:
-      exit_code: ${{ steps.lychee.outputs.exit_code }}
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v6
-        with:
-          # workflow_dispatch can run from any ref, but the report is
-          # repository-global. Always measure the default branch so a manual
-          # run from a topic branch cannot close a report that main warrants,
-          # or overwrite it with branch-only findings.
-          ref: ${{ github.event.repository.default_branch }}
-          persist-credentials: false
-
-      - name: Check links
-        id: lychee
-        uses: lycheeverse/lychee-action@e7477775783ea5526144ba13e8db5eec57747ce8 # v2.9.0
-        with:
-          # Restricted to http(s) on purpose. Much of docs/src is generated
-          # API reference (the js/ tree comes from `npm run docs` in nodejs)
-          # and the hand-written pages use mkdocstrings cross-references and
-          # nav-relative paths that only resolve in the site mkdocs builds,
-          # not in this checkout, so relative links would be reported as
-          # broken on every run.
-          args: >-
-            --scheme https
-            --scheme http
-            --no-progress
-            --max-retries 3
-            --timeout 20
-            'docs/src/**/*.md'
-          format: json
-          output: ./lychee/out.json
-          jobSummary: false
-          # The report, not a red build, is the signal for broken links. The
-          # validation step below still fails the run if the check itself
-          # breaks.
-          fail: false
-
-      - name: Validate report
-        # lychee does not reserve exit code 2 for broken links: its CLI
-        # parser also exits 2 on an invalid option, before any link was
-        # checked or any report written. Only a parseable report whose
-        # counts agree with the exit code counts as a link verdict; anything
-        # else fails here, and the report job below is skipped entirely, so
-        # the tracking issue is never touched. Exit 2 covers timeouts as
-        # well as errors, and a timed-out host is exactly the transient
-        # unavailability this report exists to surface, so both count as
-        # findings. Requiring total > 0 also catches a glob that silently
-        # stopped matching any file.
-        if: steps.lychee.outputs.exit_code == 0 || steps.lychee.outputs.exit_code == 2
-        env:
-          EXIT_CODE: ${{ steps.lychee.outputs.exit_code }}
-        run: |
-          jq -e --argjson code "$EXIT_CODE" '
-            (.total > 0) and
-            (if $code == 0
-             then .errors == 0 and .timeouts == 0
-               and (.error_map | length == 0) and (.timeout_map | length == 0)
-             else (.errors + .timeouts) > 0
-               and ((.error_map | length) + (.timeout_map | length)) > 0
-             end)
-          ' ./lychee/out.json
-
-      - name: Upload report
-        if: steps.lychee.outputs.exit_code == 2
-        uses: actions/upload-artifact@v7
-        with:
-          name: link-report
-          path: ./lychee/out.json
-          retention-days: 7
-
-  report:
-    name: Update report issue
-    needs: scan
-    runs-on: ubuntu-24.04
-    # Deliberately no checkout: this job needs the report artifact and the
-    # issues API, not the repository contents.
-    permissions:
-      issues: write
-    env:
-      EXIT_CODE: ${{ needs.scan.outputs.exit_code }}
-      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-    steps:
-      - name: Classify checker result
-        # lychee exits 0 when every link resolves and 2 when links fail,
-        # both already cross-checked against the report by the scan job's
-        # validation step. Anything else (1 runtime, 3 bad config) means the
-        # check never produced a link verdict, which must surface as a failed
-        # run rather than be published as "broken documentation links".
-        run: |
-          case "$EXIT_CODE" in
-            0|2)
-              echo "lychee exit code $EXIT_CODE"
-              ;;
-            *)
-              echo "::error::lychee exited with '$EXIT_CODE': the link check did not complete. Leaving the report issue untouched."
-              exit 1
-              ;;
-          esac
-
-      - name: Find existing report issue
-        id: report
-        # Matched on title alone, and through search rather than a listing:
-        # the issue action applies labels in a separate call after creating the
-        # issue, so a label filter misses a half-created report, and this
-        # repository has far more open issues than one listing page holds.
-        # Closed issues are included because a healthy run closes the report:
-        # an open-only lookup would forget that identity and the next failing
-        # run would open a duplicate. The oldest match stays the canonical
-        # report and is reopened below when links break again.
-        run: |
-          match=$(gh issue list --repo "$GITHUB_REPOSITORY" --state all \
-            --search "in:title \"$REPORT_TITLE\" author:app/github-actions" \
-            --limit 50 --json number,title,state \
-            --jq "[.[] | select(.title == \"$REPORT_TITLE\")] | sort_by(.number) | first // empty")
-          echo "number=$(jq -r '.number // empty' <<<"$match")" >> "$GITHUB_OUTPUT"
-          echo "state=$(jq -r '.state // empty' <<<"$match")" >> "$GITHUB_OUTPUT"
-
-      - name: Download report
-        if: env.EXIT_CODE == 2
-        uses: actions/download-artifact@v8
-        with:
-          name: link-report
-          path: ./lychee
-
-      - name: Compose report
-        if: env.EXIT_CODE == 2
-        run: |
-          run_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
-          {
-            echo "Broken documentation links found by [\`$GITHUB_WORKFLOW\`]($run_url)."
-            echo
-            echo "This issue is rewritten by every scheduled run and closed automatically once all links resolve."
-            echo
-            echo "Entries can be false positives: some sites rate-limit or block automated clients while working fine in a browser. Confirm before editing the docs, and add persistent offenders to \`--exclude\` in \`.github/workflows/docs-link-check.yml\`."
-            echo
-            # Timeouts are reported alongside errors: entries land in
-            # timeout_map with a status text instead of an HTTP code.
-            jq -r '
-              "\(.errors) of \(.total) links failed, \(.timeouts) timed out.",
-              "",
-              ([(.error_map | to_entries[]), (.timeout_map | to_entries[])]
-               | group_by(.key)[] |
-                "### Errors in \(.[0].key)",
-                "",
-                (map(.value[])[] | "* [\(.status.code // .status.text // "ERR")] <\(.url)> — \(.status.details // .status.text // "unknown error")"),
-                "")
-            ' ./lychee/out.json
-          } > ./lychee/issue.md
-
-      - name: Reopen report issue
-        # A healthy run closes the report, and the issue action below only
-        # rewrites the body of whatever number it is given. Without an
-        # explicit reopen, the 2 -> 0 -> 2 sequence would keep rewriting a
-        # closed issue while links are broken. A CLOSED state implies the
-        # lookup found a canonical issue, so no separate emptiness check.
-        if: env.EXIT_CODE == 2 && steps.report.outputs.state == 'CLOSED'
-        env:
-          ISSUE_NUMBER: ${{ steps.report.outputs.number }}
-        run: |
-          run_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
-          gh issue reopen "$ISSUE_NUMBER" --repo "$GITHUB_REPOSITORY" \
-            --comment "Broken documentation links found again in [the latest run]($run_url)."
-
-      - name: Report broken links
-        if: env.EXIT_CODE == 2
-        uses: peter-evans/create-issue-from-file@fca9117c27cdc29c6c4db3b86c48e4115a786710 # v6.0.0
-        with:
-          # Empty on the first failing run, which creates the issue; afterwards
-          # the same issue is updated in place.
-          issue-number: ${{ steps.report.outputs.number }}
-          title: ${{ env.REPORT_TITLE }}
-          content-filepath: ./lychee/issue.md
-          labels: documentation
-
-      - name: Close report issue once links are healthy
-        # An OPEN state implies the lookup found a canonical issue; a report
-        # that is already closed needs nothing.
-        if: env.EXIT_CODE == 0 && steps.report.outputs.state == 'OPEN'
-        env:
-          ISSUE_NUMBER: ${{ steps.report.outputs.number }}
-        run: |
-          run_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
-          gh issue close "$ISSUE_NUMBER" --repo "$GITHUB_REPOSITORY" \
-            --comment "All documentation links resolved in [the latest run]($run_url)."
@@ -296,18 +296,16 @@ jobs:
          cargo update -p aws-types --precise 1.3.9
          cargo update -p aws-sigv4 --precise 1.3.5
          cargo update -p aws-credential-types --precise 1.2.8
-          # aws-smithy-checksums must stay at or above 0.63.13: OpenDAL's S3
-          # service needs crc-fast ~1.9, and older releases pin it to ~1.3.
-          cargo update -p aws-smithy-checksums --precise 0.63.13
+          cargo update -p aws-smithy-checksums --precise 0.63.9
          cargo update -p aws-smithy-runtime --precise 1.9.3
-          cargo update -p aws-smithy-http --precise 0.62.6
-          cargo update -p aws-smithy-eventstream --precise 0.60.14
+          cargo update -p aws-smithy-http --precise 0.62.4
+          cargo update -p aws-smithy-eventstream --precise 0.60.12
          cargo update -p aws-smithy-http-client --precise 1.1.3
          cargo update -p aws-smithy-observability --precise 0.1.4
          cargo update -p aws-smithy-query --precise 0.60.8
-          cargo update -p aws-smithy-runtime-api --precise 1.9.3
-          cargo update -p aws-smithy-async --precise 1.2.7
-          cargo update -p aws-smithy-types --precise 1.3.6
+          cargo update -p aws-smithy-runtime-api --precise 1.9.1
+          cargo update -p aws-smithy-async --precise 1.2.6
+          cargo update -p aws-smithy-types --precise 1.3.5
          cargo update -p aws-smithy-xml --precise 0.60.11
          cargo update -p home --precise 0.5.9
      - name: cargo +${{ matrix.msrv }} check
@@ -601,7 +601,7 @@ dependencies = [
 "bytes",
 "fastrand",
 "hex",
- "http 1.5.0",
+ "http 1.4.2",
 "sha1 0.10.6",
 "time",
 "tokio",
@@ -664,7 +664,7 @@ dependencies = [
 "bytes-utils",
 "fastrand",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 0.4.6",
 "http-body 1.1.0",
 "percent-encoding",
@@ -694,7 +694,7 @@ dependencies = [
 "bytes",
 "fastrand",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body-util",
 "regex-lite",
 "tracing",
@@ -719,7 +719,7 @@ dependencies = [
 "bytes",
 "fastrand",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "regex-lite",
 "tracing",
 ]
@@ -743,7 +743,7 @@ dependencies = [
 "bytes",
 "fastrand",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "regex-lite",
 "tracing",
 ]
@@ -773,7 +773,7 @@ dependencies = [
 "hex",
 "hmac 0.13.0",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "lru 0.16.4",
 "percent-encoding",
@@ -802,7 +802,7 @@ dependencies = [
 "bytes",
 "fastrand",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "regex-lite",
 "tracing",
 ]
@@ -826,7 +826,7 @@ dependencies = [
 "bytes",
 "fastrand",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "regex-lite",
 "tracing",
 ]
@@ -851,7 +851,7 @@ dependencies = [
 "aws-types",
 "fastrand",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "regex-lite",
 "tracing",
 ]
@@ -873,7 +873,7 @@ dependencies = [
 "hex",
 "hmac 0.13.0",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "p256",
 "percent-encoding",
 "ring",
@@ -906,7 +906,7 @@ dependencies = [
 "bytes",
 "crc-fast",
 "hex",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "http-body-util",
 "md-5 0.11.0",
@@ -940,7 +940,7 @@ dependencies = [
 "bytes-utils",
 "futures-core",
 "futures-util",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "http-body-util",
 "percent-encoding",
@@ -961,7 +961,7 @@ dependencies = [
 "h2 0.3.27",
 "h2 0.4.14",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 0.4.6",
 "hyper 0.14.32",
 "hyper 1.9.0",
@@ -1023,7 +1023,7 @@ dependencies = [
 "bytes",
 "fastrand",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 0.4.6",
 "http-body 1.1.0",
 "http-body-util",
@@ -1044,7 +1044,7 @@ dependencies = [
 "aws-smithy-types",
 "bytes",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "pin-project-lite",
 "tokio",
 "tracing",
@@ -1070,7 +1070,7 @@ checksum = "7442cb268338f0eb8278140a107c046756aa01093d8ef5e99628d34ae09c94f5"
 dependencies = [
 "aws-smithy-runtime-api",
 "aws-smithy-types",
- "http 1.5.0",
+ "http 1.4.2",
 ]

 [[package]]
@@ -1084,7 +1084,7 @@ dependencies = [
 "bytes-utils",
 "futures-core",
 "http 0.2.12",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 0.4.6",
 "http-body 1.1.0",
 "http-body-util",
@@ -1132,7 +1132,7 @@ dependencies = [
 "axum-core",
 "bytes",
 "futures-util",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "http-body-util",
 "hyper 1.9.0",
@@ -1165,7 +1165,7 @@ dependencies = [
 "async-trait",
 "bytes",
 "futures-util",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "http-body-util",
 "mime",
@@ -1243,9 +1243,9 @@ checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"

 [[package]]
 name = "base64"
-version = "0.23.1"
+version = "0.23.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
+checksum = "b25655df2c3cdd83c5e5b293b88acd880332b2ddadd7c30ac43144fdc0033da9"

 [[package]]
 name = "base64-simd"
@@ -3441,12 +3441,6 @@ dependencies = [
 "percent-encoding",
 ]

-[[package]]
-name = "frostem"
-version = "1.20260804.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "82eb03a32a1d50555353c85a7b9d3279a6f1e91af9890b789acdf544ed57c8d7"
-
 [[package]]
 name = "fs_extra"
 version = "1.3.0"
@@ -3455,8 +3449,8 @@ checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c"

 [[package]]
 name = "fsst"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-array",
 "rand 0.9.5",
@@ -3813,21 +3807,20 @@ dependencies = [

 [[package]]
 name = "goosefs-sdk"
-version = "0.1.9"
+version = "0.1.8"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e1ea4eee6dcbc31b25ab4fd577adc55b677d2bed3aa3016c44c58fbe1b2298a5"
+checksum = "4a9bc9414e3b2cb0bd08dfe0eb315b177e86b119c7fa5e16179c92fc7b184860"
 dependencies = [
 "arc-swap",
 "async-trait",
 "bytes",
 "dashmap",
- "fastrand",
 "futures",
 "hostname",
 "io-uring",
 "itoa",
 "libc",
- "lru 0.18.2",
+ "lru 0.12.5",
 "memmap2 0.9.10",
 "moka",
 "prost",
@@ -3886,7 +3879,7 @@ dependencies = [
 "fnv",
 "futures-core",
 "futures-sink",
- "http 1.5.0",
+ "http 1.4.2",
 "indexmap 2.14.0",
 "slab",
 "tokio",
@@ -3932,6 +3925,8 @@ version = "0.15.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
 dependencies = [
+ "allocator-api2",
+ "equivalent",
 "foldhash 0.1.5",
 ]

@@ -3997,7 +3992,7 @@ checksum = "629d8f3bbeda9d148036d6b0de0a3ab947abd08ce90626327fc3547a49d59d97"
 dependencies = [
 "dirs",
 "futures",
- "http 1.5.0",
+ "http 1.4.2",
 "indicatif",
 "libc",
 "log",
@@ -4020,7 +4015,7 @@ checksum = "430b33fa84f92796d4d263070b6c0d3ca219df7b9a0e1853ee431029b1612bcd"
 dependencies = [
 "async-trait",
 "bytes",
- "http 1.5.0",
+ "http 1.4.2",
 "more-asserts",
 "serde",
 "thiserror 2.0.18",
@@ -4085,9 +4080,9 @@ dependencies = [

 [[package]]
 name = "http"
-version = "1.5.0"
+version = "1.4.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "918d3568bebf352712bc2ef3d46a8bcf1a75b373be6539de198e9105cbbf9ce0"
+checksum = "6970f50e31d6fc17d3fa27329444bfa74e196cf62e95052a3f6fee181dba6425"
 dependencies = [
 "bytes",
 "itoa",
@@ -4111,7 +4106,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "ca2a8f2913ee65f60facd6a5905613afaa448497a0230cc41ce022d93290bc2c"
 dependencies = [
 "bytes",
- "http 1.5.0",
+ "http 1.4.2",
 ]

 [[package]]
@@ -4122,7 +4117,7 @@ checksum = "b021d93e26becf5dc7e1b75b1bed1fd93124b374ceb73f43d4d4eafec896a64a"
 dependencies = [
 "bytes",
 "futures-core",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "pin-project-lite",
 ]
@@ -4189,7 +4184,7 @@ dependencies = [
 "futures-channel",
 "futures-core",
 "h2 0.4.14",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "httparse",
 "httpdate",
@@ -4221,7 +4216,7 @@ version = "0.27.9"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "33ca68d021ef39cf6463ab54c1d0f5daf03377b70561305bb89a8f83aab66e0f"
 dependencies = [
- "http 1.5.0",
+ "http 1.4.2",
 "hyper 1.9.0",
 "hyper-util",
 "rustls 0.23.40",
@@ -4255,7 +4250,7 @@ dependencies = [
 "bytes",
 "futures-channel",
 "futures-util",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "hyper 1.9.0",
 "ipnet",
@@ -4815,8 +4810,8 @@ checksum = "e037a2e1d8d5fdbd49b16a4ea09d5d6401c1f29eca5ff29d03d3824dba16256a"

 [[package]]
 name = "lance"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arc-swap",
 "arrow",
@@ -4890,8 +4885,8 @@ dependencies = [

 [[package]]
 name = "lance-arrow"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-array",
 "arrow-buffer",
@@ -4913,7 +4908,7 @@ dependencies = [
 [[package]]
 name = "lance-arrow-scalar"
 version = "58.0.0"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-array",
 "arrow-buffer",
@@ -4927,7 +4922,7 @@ dependencies = [
 [[package]]
 name = "lance-arrow-stats"
 version = "58.0.0"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-array",
 "arrow-schema",
@@ -4936,8 +4931,8 @@ dependencies = [

 [[package]]
 name = "lance-bitpacking"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrayref",
 "crunchy",
@@ -4947,8 +4942,8 @@ dependencies = [

 [[package]]
 name = "lance-core"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-array",
 "arrow-buffer",
@@ -4988,8 +4983,8 @@ dependencies = [

 [[package]]
 name = "lance-datafusion"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow",
 "arrow-array",
@@ -5019,8 +5014,8 @@ dependencies = [

 [[package]]
 name = "lance-datagen"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow",
 "arrow-array",
@@ -5037,8 +5032,8 @@ dependencies = [

 [[package]]
 name = "lance-derive"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "proc-macro2",
 "quote",
@@ -5047,8 +5042,8 @@ dependencies = [

 [[package]]
 name = "lance-encoding"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-arith",
 "arrow-array",
@@ -5074,6 +5069,7 @@ dependencies = [
 "prost",
 "prost-build",
 "rand 0.9.5",
+ "strum 0.26.3",
 "tokio",
 "tracing",
 "xxhash-rust",
@@ -5082,8 +5078,8 @@ dependencies = [

 [[package]]
 name = "lance-file"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-arith",
 "arrow-array",
@@ -5114,8 +5110,8 @@ dependencies = [

 [[package]]
 name = "lance-index"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arc-swap",
 "arrow",
@@ -5182,8 +5178,8 @@ dependencies = [

 [[package]]
 name = "lance-index-core"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-array",
 "arrow-schema",
@@ -5205,8 +5201,8 @@ dependencies = [

 [[package]]
 name = "lance-io"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow",
 "arrow-array",
@@ -5218,7 +5214,7 @@ dependencies = [
 "bytes",
 "chrono",
 "futures",
- "http 1.5.0",
+ "http 1.4.2",
 "io-uring",
 "lance-arrow",
 "lance-core",
@@ -5242,8 +5238,8 @@ dependencies = [

 [[package]]
 name = "lance-linalg"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-array",
 "arrow-buffer",
@@ -5259,8 +5255,8 @@ dependencies = [

 [[package]]
 name = "lance-namespace"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow",
 "async-trait",
@@ -5272,8 +5268,8 @@ dependencies = [

 [[package]]
 name = "lance-namespace-impls"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow",
 "arrow-ipc",
@@ -5327,8 +5323,8 @@ dependencies = [

 [[package]]
 name = "lance-select"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-array",
 "arrow-buffer",
@@ -5343,8 +5339,8 @@ dependencies = [

 [[package]]
 name = "lance-table"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow",
 "arrow-array",
@@ -5383,8 +5379,8 @@ dependencies = [

 [[package]]
 name = "lance-testing"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
 "arrow-array",
 "arrow-schema",
@@ -5397,13 +5393,13 @@ dependencies = [

 [[package]]
 name = "lance-tokenizer"
-version = "11.0.0-beta.2"
-source = "git+https://github.com/lance-format/lance.git?tag=v11.0.0-beta.2#35da5d920159b49d1b53032652f7615ab699c160"
+version = "10.1.0-beta.2"
+source = "git+https://github.com/lance-format/lance.git?tag=v10.1.0-beta.2#94cca93d13b1c731afd6660d46ad2f8d4b4345c4"
 dependencies = [
- "frostem",
 "icu_segmenter",
 "jieba-rs",
 "lindera",
+ "rust-stemmers",
 "serde",
 "stop-words",
 "unicode-normalization",
@@ -5450,7 +5446,7 @@ dependencies = [
 "goosefs-sdk",
 "half",
 "hf-hub",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "lance",
 "lance-arrow",
@@ -5681,7 +5677,7 @@ dependencies = [
 "serde",
 "serde_json",
 "serde_yaml_ng",
- "strum",
+ "strum 0.28.0",
 "strum_macros 0.28.0",
 "unicode-blocks",
 "unicode-normalization",
@@ -5711,7 +5707,7 @@ dependencies = [
 "rkyv",
 "serde",
 "serde_json",
- "strum",
+ "strum 0.28.0",
 "strum_macros 0.28.0",
 "thiserror 2.0.18",
 ]
@@ -5774,6 +5770,15 @@ dependencies = [
 "tracing-subscriber",
 ]

+[[package]]
+name = "lru"
+version = "0.12.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "234cf4f4a04dc1f57e24b96cc0cd600cf2af460d4161ac5ecdd0af8e1f3b2a38"
+dependencies = [
+ "hashbrown 0.15.5",
+]
+
 [[package]]
 name = "lru"
 version = "0.16.4"
@@ -5783,15 +5788,6 @@ dependencies = [
 "hashbrown 0.16.1",
 ]

-[[package]]
-name = "lru"
-version = "0.18.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5d2f2f9b4ba7e6b24d95e7e899329d35be83bcded72c8540cdd5368932d1d90a"
-dependencies = [
- "hashbrown 0.17.1",
-]
-
 [[package]]
 name = "lru-slab"
 version = "0.1.2"
@@ -6125,15 +6121,15 @@ dependencies = [

 [[package]]
 name = "napi-build"
-version = "2.4.0"
+version = "2.3.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5282704fbe8d49b0cf8b08e3f33233416a528658f205c7e5ace63b582de0b11c"
+checksum = "c9c366d2c8c60b86fa632df75f745509b52f9128f91a6bad4c796e44abb505e1"

 [[package]]
 name = "napi-derive"
-version = "3.6.1"
+version = "3.6.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "4d5c9c02556ea6dc99dffd36c1ce60141411657438501a125b675776d011ce92"
+checksum = "a49c513341a61a16a10af6efcce46b30d0822ba2d4fb197d24d33dfc199c78d5"
 dependencies = [
 "convert_case",
 "ctor 1.0.12",
@@ -6145,9 +6141,9 @@ dependencies = [

 [[package]]
 name = "napi-derive-backend"
-version = "6.1.1"
+version = "6.0.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d60b5d773ad46c698c8cc2cd9fde0b283d39cbb7f71c04bee633c7bdba4423bd"
+checksum = "4747005fa3e2c9989ac45a723a514c5db2411238b72981a3cda4c701a9dfea17"
 dependencies = [
 "convert_case",
 "proc-macro2",
@@ -6401,7 +6397,7 @@ dependencies = [
 "futures-channel",
 "futures-core",
 "futures-util",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body-util",
 "httparse",
 "humantime",
@@ -6520,10 +6516,10 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "ec75551ff4cf3e57da98979f6a937aaa9ddb3915bf68cc17d03df733be6646ed"
 dependencies = [
 "anyhow",
- "base64 0.23.1",
+ "base64 0.23.0",
 "bytes",
 "futures",
- "http 1.5.0",
+ "http 1.4.2",
 "jiff",
 "log",
 "md-5 0.11.0",
@@ -6547,7 +6543,7 @@ checksum = "ad4d4f19c3ce01126a30611f8e544eaa217104a278c889ac17c9374fe4f9e4ef"
 dependencies = [
 "bytes",
 "futures",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "opendal-core",
 "reqwest 0.13.4",
@@ -6560,7 +6556,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "249ac5b0aa5a7a6c3737342d10456067937f9c9a6f3f02544271f7908ab91081"
 dependencies = [
 "futures",
- "http 1.5.0",
+ "http 1.4.2",
 "mea",
 "opendal-core",
 ]
@@ -6602,9 +6598,9 @@ version = "0.58.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "b3310fbbb48f111c6f590473c2cd15e1b7f8e384444b0d4e328f0464c864d767"
 dependencies = [
- "base64 0.23.1",
+ "base64 0.23.0",
 "bytes",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "opendal-core",
 "opendal-service-azure-common",
@@ -6623,9 +6619,9 @@ version = "0.58.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "2e3c406729935fe214ce574d68681a1ff7e0b322548f14094912bdbfe50e5c53"
 dependencies = [
- "base64 0.23.1",
+ "base64 0.23.0",
 "bytes",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "mea",
 "opendal-core",
@@ -6644,7 +6640,7 @@ version = "0.58.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "7348c88edf15af435b7be930077746b569fac5e738c1bf6a363b675e7317c9df"
 dependencies = [
- "http 1.5.0",
+ "http 1.4.2",
 "opendal-core",
 ]

@@ -6655,7 +6651,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "d533d4582105d269c8aebeee5f0e8bcf960f41b8aab6197df7012254d9f39bf0"
 dependencies = [
 "bytes",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "opendal-core",
 "quick-xml 0.41.0",
@@ -6673,7 +6669,7 @@ checksum = "007f3fba63c21e516c956b891e96ff9892d8175662bfb781cdada9d3766a11e6"
 dependencies = [
 "async-trait",
 "bytes",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "opendal-core",
 "percent-encoding",
@@ -6708,7 +6704,7 @@ checksum = "b41fd41eb7ed03c5e66cefda61e8e117808ffd2908f2916737cb020a6beb02c7"
 dependencies = [
 "bytes",
 "hf-xet",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "opendal-core",
 "percent-encoding",
@@ -6723,7 +6719,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "cd528ec2d49c5ca69e674ffed7b3e0686fb9cfcfea0596870de381467fda4f1b"
 dependencies = [
 "bytes",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "opendal-core",
 "quick-xml 0.41.0",
@@ -6739,10 +6735,10 @@ version = "0.58.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "58e80cdf192d7eff05feed747894d64f81905ac4eaf132edf7ea270abdd2d663"
 dependencies = [
- "base64 0.23.1",
+ "base64 0.23.0",
 "bytes",
 "crc-fast",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "md-5 0.11.0",
 "opendal-core",
@@ -8288,13 +8284,13 @@ dependencies = [

 [[package]]
 name = "reqsign-aliyun-oss"
-version = "3.1.3"
+version = "3.1.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a5e6d659fcdbca6fe2d7ef109c2e28499b7be80501f1bb86c10caf5ec8ac1219"
+checksum = "9c0f9f69a519dd6958c4b43606bb8e1278cdc76d611fc8fed4b796eee548dc0f"
 dependencies = [
 "anyhow",
 "form_urlencoded",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "percent-encoding",
 "reqsign-core",
@@ -8304,15 +8300,16 @@ dependencies = [
 ]

 [[package]]
-name = "reqsign-aws-core"
+name = "reqsign-aws-v4"
 version = "3.0.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e4af084e1f3cbf3e67e0c972765399bce54ecec804cceba46b39a8331f3c1bff"
+checksum = "cc883bc56889f3e4a419265c87facea222a921debc5c6f15c7fd8b68ec4b36b2"
 dependencies = [
+ "anyhow",
 "bytes",
 "form_urlencoded",
 "hex",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "percent-encoding",
 "quick-xml 0.41.0",
@@ -8324,32 +8321,17 @@ dependencies = [
 "sha1 0.11.0",
 ]

-[[package]]
-name = "reqsign-aws-v4"
-version = "3.1.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "4ac5b3b7cefa28933792b439186459f77f19f9b6edbeab41b8b187150361a206"
-dependencies = [
- "bytes",
- "http 1.5.0",
- "log",
- "quick-xml 0.41.0",
- "reqsign-aws-core",
- "reqsign-core",
- "serde",
-]
-
 [[package]]
 name = "reqsign-azure-storage"
-version = "3.1.2"
+version = "3.1.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "2824e7da3c2cc42ac3406c674eb57c89127fdcd97f3a73c608cfc680505ea134"
+checksum = "a6ebd8524185ce9c64063e3095f83968acfa90922f00c601a4a0f3aca15b077e"
 dependencies = [
 "anyhow",
- "base64 0.23.1",
+ "base64 0.22.1",
 "bytes",
 "form_urlencoded",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "pem",
 "percent-encoding",
@@ -8362,17 +8344,17 @@ dependencies = [

 [[package]]
 name = "reqsign-core"
-version = "3.2.1"
+version = "3.2.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "c07dd510b1e1b9b241883e483358147fb2ed2d497a7b39b065ba61eb93deceb0"
+checksum = "7e38b44697c60a823705ccef85cb04d8e0527c9d16ed7c58bf1c6395bdd24ceb"
 dependencies = [
 "anyhow",
- "base64 0.23.1",
+ "base64 0.22.1",
 "bytes",
 "futures",
 "hex",
 "hmac 0.13.0",
- "http 1.5.0",
+ "http 1.4.2",
 "jiff",
 "log",
 "percent-encoding",
@@ -8386,9 +8368,9 @@ dependencies = [

 [[package]]
 name = "reqsign-file-read-tokio"
-version = "3.0.4"
+version = "3.0.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "663d9d55abd0df0830ef0ae43708297cc1371cf4e8ca91f3ac813c309cca8c98"
+checksum = "688ff0ae421b8d4b92b53fdafaf53df2de28f428a9962edcf21702990b26f74b"
 dependencies = [
 "anyhow",
 "reqsign-core",
@@ -8397,12 +8379,12 @@ dependencies = [

 [[package]]
 name = "reqsign-google"
-version = "3.0.4"
+version = "3.0.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "4080a227f82a09f68540ecd028622065d7ac4c0bcb8727a25bdcfc0526235792"
+checksum = "a96da0b579b846d358090cb06b9e3c2ad1375529efbe3e0c45f96bd7bcf043ea"
 dependencies = [
 "form_urlencoded",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "percent-encoding",
 "reqsign-aws-v4",
@@ -8415,12 +8397,12 @@ dependencies = [

 [[package]]
 name = "reqsign-tencent-cos"
-version = "3.0.4"
+version = "3.0.3"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "764629c90f7c3566a6d4e4641ebab9acd604ce02e16eda4d37c7d7e79e16ed90"
+checksum = "f6497dd9f6e3d1349b420521484099b284f95e8d3a65f088fccef42493a7b644"
 dependencies = [
 "anyhow",
- "http 1.5.0",
+ "http 1.4.2",
 "log",
 "percent-encoding",
 "reqsign-core",
@@ -8440,7 +8422,7 @@ dependencies = [
 "futures-core",
 "futures-util",
 "h2 0.4.14",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "http-body-util",
 "hyper 1.9.0",
@@ -8484,7 +8466,7 @@ dependencies = [
 "bytes",
 "futures-core",
 "futures-util",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "http-body-util",
 "hyper 1.9.0",
@@ -8538,7 +8520,7 @@ checksum = "199dda04a536b532d0cc04d7979e39b1c763ea749bf91507017069c00b96056f"
 dependencies = [
 "anyhow",
 "async-trait",
- "http 1.5.0",
+ "http 1.4.2",
 "reqwest 0.13.4",
 "thiserror 2.0.18",
 "tower-service",
@@ -8580,9 +8562,9 @@ dependencies = [

 [[package]]
 name = "rkyv"
-version = "0.8.17"
+version = "0.8.16"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "815cc8a37159a463064825246cadb07961e25cd9885908606f6d08a98d8f8874"
+checksum = "73389e0c99e664f919275ab5b5b0471391fe9a8de61e1dff9b1eaf56a90f16e3"
 dependencies = [
 "bytecheck",
 "bytes",
@@ -8599,9 +8581,9 @@ dependencies = [

 [[package]]
 name = "rkyv_derive"
-version = "0.8.17"
+version = "0.8.16"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "c0ed1a78a1b19d184b0daa629dd9a024573173ec7d485b287cb369fb3607cc1c"
+checksum = "5d2ed0b54125315fb36bd021e82d314d1c126548f871634b483f46b31d13cac6"
 dependencies = [
 "proc-macro2",
 "quote",
@@ -8679,6 +8661,16 @@ dependencies = [
 "ordered-multimap",
 ]

+[[package]]
+name = "rust-stemmers"
+version = "1.2.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "e46a2036019fdb888131db7a4c847a1063a7493f971ed94ea82c67eada63ca54"
+dependencies = [
+ "serde",
+ "serde_derive",
+]
+
 [[package]]
 name = "rustc-demangle"
 version = "0.1.27"
@@ -9552,6 +9544,15 @@ version = "0.11.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f"

+[[package]]
+name = "strum"
+version = "0.26.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "8fec0f0aef304996cf250b31b5a10dee7980c85da9d759361292b8bca5a18f06"
+dependencies = [
+ "strum_macros 0.26.4",
+]
+
 [[package]]
 name = "strum"
 version = "0.28.0"
@@ -9574,6 +9575,19 @@ dependencies = [
 "syn 2.0.117",
 ]

+[[package]]
+name = "strum_macros"
+version = "0.26.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4c6bee85a5a24955dc440386795aa378cd9cf82acd5f764469152d2270e581be"
+dependencies = [
+ "heck 0.5.0",
+ "proc-macro2",
+ "quote",
+ "rustversion",
+ "syn 2.0.117",
+]
+
 [[package]]
 name = "strum_macros"
 version = "0.28.0"
@@ -10096,7 +10110,7 @@ dependencies = [
 "base64 0.22.1",
 "bytes",
 "h2 0.4.14",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "http-body-util",
 "hyper 1.9.0",
@@ -10152,7 +10166,7 @@ checksum = "1e9cd434a998747dd2c4276bc96ee2e0c7a2eadf3cae88e52be55a05fa9053f5"
 dependencies = [
 "bitflags 2.11.1",
 "bytes",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "http-body-util",
 "pin-project-lite",
@@ -10172,7 +10186,7 @@ dependencies = [
 "bytes",
 "futures-core",
 "futures-util",
- "http 1.5.0",
+ "http 1.4.2",
 "http-body 1.1.0",
 "http-body-util",
 "pin-project-lite",
@@ -11193,7 +11207,7 @@ dependencies = [
 "clap",
 "crc32fast",
 "futures",
- "http 1.5.0",
+ "http 1.4.2",
 "hyper 1.9.0",
 "lazy_static",
 "more-asserts",
@@ -11267,7 +11281,7 @@ dependencies = [
 "chrono",
 "clap",
 "gearhash",
- "http 1.5.0",
+ "http 1.4.2",
 "itertools 0.14.0",
 "lazy_static",
 "more-asserts",
@@ -13,20 +13,20 @@ categories = ["database-implementations"]
 rust-version = "1.91.0"

 [workspace.dependencies]
-lance = { "version" = "=11.0.0-beta.2", default-features = false, "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-core = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-datagen = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-file = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-io = { "version" = "=11.0.0-beta.2", default-features = false, "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-index = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-linalg = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-namespace = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-namespace-impls = { "version" = "=11.0.0-beta.2", default-features = false, "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-table = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-testing = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-datafusion = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-encoding = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
-lance-arrow = { "version" = "=11.0.0-beta.2", "tag" = "v11.0.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance = { "version" = "=10.1.0-beta.2", default-features = false, "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-core = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-datagen = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-file = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-io = { "version" = "=10.1.0-beta.2", default-features = false, "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-index = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-linalg = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-namespace = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-namespace-impls = { "version" = "=10.1.0-beta.2", default-features = false, "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-table = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-testing = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-datafusion = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-encoding = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
+lance-arrow = { "version" = "=10.1.0-beta.2", "tag" = "v10.1.0-beta.2", "git" = "https://github.com/lance-format/lance.git" }
 ahash = "0.8"
 # Note that this one does not include pyarrow
 arrow = { version = "58.0.0", optional = false }
@@ -52,7 +52,7 @@ env_logger = "0.11"
 half = { "version" = "2.7.1", default-features = false, features = [
    "num-traits",
 ] }
-futures = "0.3"
+futures = "0"
 log = "0.4"
 metrics = "0.24"
 metrics-util = "0.19"
@@ -31,7 +31,7 @@ is also an [asynchronous API client](#connections-asynchronous).
 ## Namespaces (Synchronous)

 A namespace-backed connection resolves tables through a
-[Lance namespace](https://lance-format.github.io/lance-namespace/) service instead of
+[Lance namespace](https://lancedb.github.io/lance-namespace/) service instead of
 listing a storage directory.

 ::: lancedb.connect_namespace
@@ -28,7 +28,7 @@
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <arrow.version>15.0.0</arrow.version>
-        <lance-core.version>11.0.0-beta.2</lance-core.version>
+        <lance-core.version>10.1.0-beta.2</lance-core.version>
        <spotless.skip>false</spotless.skip>
        <spotless.version>2.30.0</spotless.version>
        <spotless.java.googlejavaformat.version>1.7</spotless.java.googlejavaformat.version>
@@ -197,35 +197,6 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
        expect(table.getChild("d")?.toJSON()).toEqual([9n, 10n, null]);
      });

-      it("will use a provided FixedSizeList schema with typed array values", function () {
-        const schema = new Schema([
-          new Field("text", new Utf8(), false),
-          new Field(
-            "vector",
-            new FixedSizeList(3, new Field("item", new Float32(), false)),
-            false,
-          ),
-        ]);
-
-        const table = makeArrowTable(
-          [
-            {
-              text: "foo",
-              vector: new Float32Array([1, 2, 3]),
-            },
-          ],
-          { schema },
-        );
-
-        expect(table.getChild("text")?.toJSON()).toEqual(["foo"]);
-        expect(
-          table
-            .getChild("vector")
-            ?.toJSON()
-            .map((value) => value.toJSON()),
-        ).toEqual([[1, 2, 3]]);
-      });
-
      it("will assume the column `vector` is FixedSizeList<Float32> by default", async function () {
        const schema = new Schema([
          new Field("a", new Float(Precision.DOUBLE), true),
@@ -11,11 +11,8 @@ import {
  Float16,
  Float32,
  Float64,
-  Int32,
  Schema,
  Utf8,
-  fromDataToBuffer,
-  tableFromIPC,
 } from "../lancedb/arrow";
 import { EmbeddingFunction, LanceSchema } from "../lancedb/embedding";
 import { getRegistry, register } from "../lancedb/embedding/registry";
@@ -187,63 +184,6 @@ describe("embedding functions", () => {
    const vector0 = JSON.parse(JSON.stringify(arr[0].vector));
    expect(vector0).toEqual([1, 2, 3]);
  });
-
-  it("should append generated vectors to a non-nullable schema", async () => {
-    @register("non_nullable_schema_test")
-    class MockEmbeddingFunction extends EmbeddingFunction<string> {
-      ndims() {
-        return 3;
-      }
-      embeddingDataType(): Float {
-        return new Float64();
-      }
-      async computeSourceEmbeddings(data: string[]) {
-        return data.map(() => [1, 2, 3]);
-      }
-    }
-
-    const schema = new Schema([
-      new Field("id", new Int32()),
-      new Field("text", new Utf8()),
-      new Field("type", new Utf8()),
-      new Field(
-        "vector",
-        new FixedSizeList(3, new Field("item", new Float64())),
-      ),
-    ]);
-    const func = new MockEmbeddingFunction();
-    const db = await connect(tmpDir.name);
-    const table = await db.createEmptyTable("test_non_nullable", schema, {
-      embeddingFunction: {
-        function: func,
-        sourceColumn: "text",
-      },
-    });
-
-    const data = [
-      { id: 1, text: "Carrot", type: "vegetable" },
-      { id: 2, text: "Apple", type: "fruit" },
-    ];
-    const buffer = await fromDataToBuffer(
-      data,
-      undefined,
-      await table.schema(),
-    );
-    const generatedTable = tableFromIPC(buffer);
-    const vectorField = generatedTable.schema.fields.find(
-      (field) => field.name === "vector",
-    );
-    expect(vectorField?.nullable).toBe(false);
-
-    await table.add(data);
-
-    const rows = await table.query().toArray();
-    expect(rows).toHaveLength(2);
-    for (const row of rows) {
-      expect([...row.vector]).toEqual([1, 2, 3]);
-    }
-  });
-
  it("should error when appending to a table with an unregistered embedding function", async () => {
    @register("mock")
    class MockEmbeddingFunction extends EmbeddingFunction<string> {
@@ -1,14 +0,0 @@
-// SPDX-License-Identifier: Apache-2.0
-// SPDX-FileCopyrightText: Copyright The LanceDB Authors
-
-import packageJson = require("../package.json");
-
-describe("package metadata", () => {
-  it("requires Node.js type declarations compatible with the runtime", () => {
-    expect(packageJson.engines.node).toBe(">= 18");
-    expect(packageJson.peerDependencies["@types/node"]).toBe(">=18");
-    expect(packageJson.peerDependenciesMeta["@types/node"]).toEqual({
-      optional: true,
-    });
-  });
-});
@@ -110,81 +110,6 @@ describe("Query outputSchema", () => {
  });
 });

-describe("Search pagination", () => {
-  let tmpDir: tmp.DirResult;
-  let table: Table;
-
-  beforeEach(async () => {
-    tmpDir = tmp.dirSync({ unsafeCleanup: true });
-    const db = await connect(tmpDir.name);
-    const schema = new Schema([
-      new Field("id", new Int64(), false),
-      new Field("text", new Utf8(), false),
-      new Field(
-        "vector",
-        new FixedSizeList(2, new Field("item", new Float32())),
-        false,
-      ),
-    ]);
-    const data = makeArrowTable(
-      [
-        { id: 1n, text: "common", vector: [0, 0] },
-        { id: 2n, text: "common common", vector: [1, 1] },
-        { id: 3n, text: "common common common", vector: [2, 2] },
-        { id: 4n, text: "common common common common", vector: [3, 3] },
-      ],
-      { schema },
-    );
-    table = await db.createTable("test", data);
-  });
-
-  afterEach(() => {
-    tmpDir.removeCallback();
-  });
-
-  it("applies offset after the vector search limit", async () => {
-    const allResults = await table
-      .vectorSearch([0, 0])
-      .select(["id"])
-      .limit(4)
-      .toArray();
-    const secondPage = await table
-      .vectorSearch([0, 0])
-      .select(["id"])
-      .limit(2)
-      .offset(2)
-      .toArray();
-
-    expect(allResults).toHaveLength(4);
-    expect(secondPage).toHaveLength(2);
-    expect(secondPage.map((row) => row.id)).toEqual(
-      allResults.slice(2, 4).map((row) => row.id),
-    );
-  });
-
-  it("applies offset after the full-text search limit", async () => {
-    await table.createIndex("text", { config: Index.fts() });
-
-    const allResults = await table
-      .search("common", "fts")
-      .select(["id"])
-      .limit(4)
-      .toArray();
-    const secondPage = await table
-      .search("common", "fts")
-      .select(["id"])
-      .limit(2)
-      .offset(2)
-      .toArray();
-
-    expect(allResults).toHaveLength(4);
-    expect(secondPage).toHaveLength(2);
-    expect(secondPage.map((row) => row.id)).toEqual(
-      allResults.slice(2, 4).map((row) => row.id),
-    );
-  });
-});
-
 describe("Query orderBy", () => {
  let tmpDir: tmp.DirResult;
  let table: Table;
@@ -170,38 +170,6 @@ describe("remote connection", () => {
    );
  });

-  it("surfaces JSON server errors from remote table operations", async () => {
-    await withMockDatabase(
-      (req, res) => {
-        const path = req.url ?? "";
-        if (path.endsWith("/describe/")) {
-          res.writeHead(200, { "Content-Type": "application/json" }).end(
-            JSON.stringify({
-              name: "broken_table",
-              version: 1,
-              schema: { fields: [] },
-            }),
-          );
-          return;
-        }
-
-        if (path.endsWith("/count_rows/")) {
-          res
-            .writeHead(400, { "Content-Type": "application/json" })
-            .end(JSON.stringify({ error: "count rows failed" }));
-          return;
-        }
-
-        res.writeHead(404).end();
-      },
-      async (db) => {
-        const table = await db.openTable("broken_table");
-
-        await expect(table.countRows()).rejects.toThrow("count rows failed");
-      },
-    );
-  });
-
  it("should pass on requested extra headers", async () => {
    await withMockDatabase(
      (req, res) => {
@@ -86,44 +86,6 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
      await expect(table.countRows()).resolves.toBe(3);
    });

-    it("should support a foreign Float64 vector schema end to end", async () => {
-      const conn = await connect(tmpDir.name);
-      const schema = new arrow.Schema([
-        new arrow.Field("resource_id", new arrow.Int32(), false),
-        new arrow.Field(
-          "vector",
-          new arrow.FixedSizeList(
-            3,
-            new arrow.Field("value", new arrow.Float64(), true),
-          ),
-          false,
-        ),
-      ]);
-      const data = [
-        {
-          // biome-ignore lint/style/useNamingConvention: matches the reported schema
-          resource_id: 0,
-          vector: [0.1, 0.1, 0.1],
-        },
-      ];
-
-      const resources = await conn.createTable("resources", data, { schema });
-
-      const existing = await resources
-        .query()
-        .where("resource_id = 0")
-        .limit(1)
-        .toArray();
-      expect(existing).toHaveLength(1);
-
-      const matched = await resources
-        .search(Float64Array.from(data[0].vector))
-        .limit(1)
-        .toArray();
-      expect(matched).toHaveLength(1);
-      expect(matched[0]["resource_id"]).toBe(0);
-    });
-
    it("should support branches", async () => {
      await table.add([{ id: 1 }]);
      expect(await table.countRows()).toBe(1);
@@ -55,13 +55,7 @@
        "openai": "4.29.2"
      },
      "peerDependencies": {
-        "@types/node": ">=18",
        "apache-arrow": ">=15.0.0 <=18.1.0"
-      },
-      "peerDependenciesMeta": {
-        "@types/node": {
-          "optional": true
-        }
      }
    },
    "node_modules/@aws-crypto/crc32": {
@@ -101,12 +101,6 @@
    "openai": "4.29.2"
  },
  "peerDependencies": {
-    "@types/node": ">=18",
    "apache-arrow": ">=15.0.0 <=18.1.0"
-  },
-  "peerDependenciesMeta": {
-    "@types/node": {
-      "optional": true
-    }
  }
 }
@@ -339,9 +339,7 @@ impl Table {
        let transforms = NewColumnTransform::SqlExpressions(transforms);
        let res = self
            .inner_ref()?
-            .add_columns()
-            .transform(transforms)
-            .execute()
+            .add_columns(transforms, None)
            .await
            .default_error()?;
        Ok(res.into())
@@ -358,9 +356,7 @@ impl Table {
        let transforms = NewColumnTransform::AllNulls(schema);
        let res = self
            .inner_ref()?
-            .add_columns()
-            .transform(transforms)
-            .execute()
+            .add_columns(transforms, None)
            .await
            .default_error()?;
        Ok(res.into())
@@ -26,7 +26,7 @@ lance-namespace-impls.workspace = true
 lance-io.workspace = true
 env_logger.workspace = true
 log.workspace = true
-pyo3 = { version = "0.28", features = ["extension-module", "abi3-py310", "chrono"] }
+pyo3 = { version = "0.28", features = ["extension-module", "abi3-py39", "chrono"] }
 chrono = { version = "0.4", default-features = false, features = ["clock"] }
 pyo3-async-runtimes = { version = "0.28", features = [
    "attributes",
@@ -43,7 +43,7 @@ libc = "0.2"
 [build-dependencies]
 pyo3-build-config = { version = "0.28", features = [
    "extension-module",
-    "abi3-py310",
+    "abi3-py39",
 ] }

 [features]
@@ -60,7 +60,7 @@ tests = [
    "pytest-asyncio>=0.21",
    "duckdb>=0.9.0",
    "pytz>=2023.3",
-    "polars>=0.19, <=1.32.3",
+    "polars>=0.19, <=1.3.0",
    "pyarrow<25",
    "pyarrow-stubs>=16.0",
    "pylance==9.0.0rc1",
@@ -140,7 +140,6 @@ include = [
    "python/lancedb/remote/errors.py",
    "python/lancedb/embeddings/__init__.py",
    "python/lancedb/_lancedb.pyi",
-    "python/type_tests/connect.py",
 ]
 exclude = ["python/tests/"]
 pythonVersion = "3.13"
@@ -707,9 +707,6 @@ class LanceDBConnection(DBConnection):
        self._namespace_client_properties = namespace_client_properties
        if _inner is not None:
            self._conn = _inner
-            # Native-derived wrappers resolve this in their async reconstruction
-            # path so construction never synchronously re-enters LOOP.
-            self._read_consistency_interval = read_consistency_interval
            self._cached_namespace_client = None
            return

@@ -759,14 +756,11 @@ class LanceDBConnection(DBConnection):
        # storage_options.  Also, this class really shouldn't be holding any state
        # beyond _conn.
        self._conn = AsyncConnection(LOOP.run(do_connect()))
-        # Keep property access synchronous so debugger introspection cannot wait on
-        # the background loop while that thread is suspended at a breakpoint.
-        self._read_consistency_interval = read_consistency_interval
        self._cached_namespace_client: Optional[LanceNamespace] = None

    @property
    def read_consistency_interval(self) -> Optional[timedelta]:
-        return self._read_consistency_interval
+        return LOOP.run(self._conn.get_read_consistency_interval())

    @property
    def session(self) -> Optional[Session]:
@@ -777,16 +771,8 @@ class LanceDBConnection(DBConnection):
        return self._conn.uri

    @classmethod
-    def from_inner(
-        cls,
-        inner: LanceDbConnection,
-        read_consistency_interval: Optional[timedelta],
-    ):
-        return cls(
-            None,
-            read_consistency_interval=read_consistency_interval,
-            _inner=inner,
-        )
+    def from_inner(cls, inner: LanceDbConnection):
+        return cls(None, _inner=inner)

    def __repr__(self) -> str:
        return f"{self.__class__.__name__}(uri={self._conn.uri!r})"
@@ -183,11 +183,7 @@ class EmbeddingFunction(BaseModel, ABC):
    def VectorField(self, **kwargs):
        """
        Creates a pydantic Field that can automatically annotate
-        the target vector column for this embedding function.
-
-        The field can be annotated as ``list[float]`` for compatibility with
-        static type checkers. LanceDB will infer the fixed vector dimension from
-        this embedding function.
+        the target vector column for this embedding function
        """
        return Field(json_schema_extra={"vector_column_for": self}, **kwargs)

@@ -101,7 +101,8 @@ class InstructorEmbeddingFunction(TextEmbeddingFunction):

    @weak_lru(maxsize=1)
    def ndims(self):
-        return len(self.generate_embeddings([[self.source_instruction, "foo"]])[0])
+        model = self.get_model()
+        return model.encode("foo").shape[0]

    def compute_query_embeddings(self, query: str, *args, **kwargs) -> List[np.array]:
        return self.generate_embeddings([[self.query_instruction, query]])
@@ -92,10 +92,8 @@ class LanceMergeInsertBuilder(object):
        self._when_not_matched_by_source_delete = True
        if isinstance(condition, Expr):
            self._when_not_matched_by_source_condition_expr = condition._inner
-            self._when_not_matched_by_source_condition = None
-        else:
+        elif condition is not None:
            self._when_not_matched_by_source_condition = condition
-            self._when_not_matched_by_source_condition_expr = None
        return self

    def use_index(self, use_index: bool) -> LanceMergeInsertBuilder:
@@ -226,7 +226,7 @@ class PermutationBuilder:

        async def do_execute():
            inner_tbl = await self._async.execute()
-            return await LanceTable.from_inner(inner_tbl)
+            return LanceTable.from_inner(inner_tbl)

        return LOOP.run(do_execute())

@@ -1 +0,0 @@
-
@@ -99,17 +99,6 @@ def Vector(
    ...     pa.field("url", pa.utf8(), False),
    ...     pa.field("embeddings", pa.list_(pa.float32(), 768))
    ... ])
-
-    Notes
-    -----
-    ``Vector`` creates a type dynamically, so calls such as ``Vector(768)`` are
-    not valid static type annotations. For an embedding field, use the standard
-    ``list[float]`` annotation when running mypy; ``VectorField`` supplies the
-    fixed dimension to LanceDB::
-
-        class MyModel(LanceModel):
-            text: str = embeddings.SourceField()
-            vector: list[float] = embeddings.VectorField()
    """

    # TODO: make a public parameterized type.
@@ -164,16 +153,6 @@ def Vector(
    return FixedSizeList


-def _raise_bare_vector_error(*_args):
-    raise TypeError("Vector must be parameterized with a dimension, e.g. Vector(128).")
-
-
-# Pydantic v1 and v2 otherwise treat the bare Vector factory as a field validator
-# and inspect its signature, which produces misleading errors about internal types.
-setattr(Vector, "__get_validators__", _raise_bare_vector_error)
-setattr(Vector, "__get_pydantic_core_schema__", _raise_bare_vector_error)
-
-
 def MultiVector(
    dim: int, value_type: pa.DataType = pa.float32(), nullable: bool = True
 ) -> Type:
@@ -390,10 +369,6 @@ def _unwrap_optional_annotation(annotation: Any) -> Any | None:

 def _pydantic_to_arrow_type(field: FieldInfo) -> pa.DataType:
    """Convert a Pydantic FieldInfo to Arrow DataType"""
-    embedding_vector_type = _embedding_vector_to_arrow_type(field)
-    if embedding_vector_type is not None:
-        return embedding_vector_type
-
    unwrapped = _unwrap_optional_annotation(field.annotation)
    if unwrapped is not None:
        return _pydantic_type_to_arrow_type(unwrapped, field)
@@ -407,32 +382,8 @@ def _pydantic_to_arrow_type(field: FieldInfo) -> pa.DataType:
    return _pydantic_type_to_arrow_type(field.annotation, field)


-def _embedding_vector_to_arrow_type(field: FieldInfo) -> pa.DataType | None:
-    """Infer a fixed-size vector type from ``VectorField`` metadata."""
-    if not _is_embedding_vector_annotation(field):
-        return None
-
-    function = get_extras(field, "vector_column_for")
-    return pa.list_(pa.float32(), function.ndims())
-
-
-def _is_embedding_vector_annotation(field: FieldInfo) -> bool:
-    if get_extras(field, "vector_column_for") is None:
-        return False
-
-    annotation = _unwrap_optional_annotation(field.annotation)
-    if annotation is None:
-        annotation = field.annotation
-
-    origin = getattr(annotation, "__origin__", None)
-    args = getattr(annotation, "__args__", ())
-    return origin is list and args == (float,)
-
-
 def is_nullable(field: FieldInfo) -> bool:
    """Check if a Pydantic FieldInfo is nullable."""
-    if _is_embedding_vector_annotation(field):
-        return True
    if _unwrap_optional_annotation(field.annotation) is not None:
        return True
    if isinstance(field.annotation, (_GenericAlias, GenericAlias)):
@@ -2697,7 +2697,7 @@ class LanceHybridQueryBuilder(LanceQueryBuilder):
            self._fts_query.phrase_query(True)
        if self._distance_type:
            self._vector_query.metric(self._distance_type)
-        if self._minimum_nprobes is not None:
+        if self._minimum_nprobes:
            self._vector_query.minimum_nprobes(self._minimum_nprobes)
        if self._maximum_nprobes is not None:
            self._vector_query.maximum_nprobes(self._maximum_nprobes)
@@ -108,11 +108,6 @@ def _should_push_down_query_table(
    return namespace_client is not None and "QueryTable" in pushdown_operations


-def _polars_predicate_pushdown_barrier(frame: Any) -> Any:
-    """Return a Polars frame unchanged while blocking predicate pushdown."""
-    return frame
-
-
 _MODEL_BACKED_TOKENIZER_PREFIXES = ("jieba", "lindera")
 _MODEL_BACKED_TOKENIZER_ERRORS = (
    "unknown base tokenizer",
@@ -869,18 +864,12 @@ class Table(ABC):
        """
        raise NotImplementedError

-    def to_polars(self, **kwargs) -> "pl.LazyFrame":
-        """Return the table as a Polars LazyFrame.
-
-        Note
-        ----
-        The Polars streaming engine is not supported because it does not currently
-        implement Python PyArrow dataset scans. Use the default engine when collecting
-        this LazyFrame.
+    def to_polars(self, **kwargs) -> "pl.DataFrame":
+        """Return the table as a polars.DataFrame.

        Returns
        -------
-        polars.LazyFrame
+        polars.DataFrame
        """
        raise NotImplementedError

@@ -1606,8 +1595,8 @@ class Table(ABC):
        Prefer this over :meth:`fetch_blobs` for large payloads. ``row_ids`` is
        a ``list[int]`` or a query ``pyarrow.Table`` carrying row identity via
        ``_rowid`` or a ``_lance_row_id`` field on the blob descriptor. Null
-        rows are ``None``. Remote tables require LanceDB Cloud server 0.5.0 or
-        newer.
+        rows are ``None``. Unsupported on LanceDB Cloud, where
+        :meth:`fetch_blobs` returns full bytes instead.
        """

    @abstractmethod
@@ -2193,15 +2182,11 @@ class LanceTable(Table):
        return self.name

    @classmethod
-    async def from_inner(cls, tbl: LanceDBTable):
-        from .db import AsyncConnection, LanceDBConnection
+    def from_inner(cls, tbl: LanceDBTable):
+        from .db import LanceDBConnection

        async_tbl = AsyncTable(tbl)
-        inner_conn = tbl.database()
-        read_consistency_interval = await AsyncConnection(
-            inner_conn
-        ).get_read_consistency_interval()
-        conn = LanceDBConnection.from_inner(inner_conn, read_consistency_interval)
+        conn = LanceDBConnection.from_inner(tbl.database())
        return cls(
            conn,
            async_tbl.name,
@@ -2580,9 +2565,6 @@ class LanceTable(Table):
        2. Currently we've disabled push-down of the filters from polars
           because polars pushdown into pyarrow uses pyarrow compute
           expressions rather than SQl strings (which LanceDB supports)
-        3. The Polars streaming engine is not supported because it does not
-           currently implement Python PyArrow dataset scans. Use the default
-           engine when collecting this LazyFrame.

        Returns
        -------
@@ -2591,12 +2573,8 @@ class LanceTable(Table):
        from lancedb.integrations.pyarrow import PyarrowDatasetAdapter

        dataset = PyarrowDatasetAdapter(self)
-        # Polars 1.32's non-PyArrow callback path passes batch_size twice.  Keep
-        # the compatible PyArrow path, but block predicates because this adapter
-        # cannot translate PyArrow expressions into LanceDB filters.
-        return pl.scan_pyarrow_dataset(dataset, batch_size=batch_size).map_batches(
-            _polars_predicate_pushdown_barrier,
-            predicate_pushdown=False,
+        return pl.scan_pyarrow_dataset(
+            dataset, allow_pyarrow_filter=False, batch_size=batch_size
        )

    # New unified API overload
@@ -395,11 +395,6 @@ def _(value: dict):
    )


-@value_to_sql.register(pa.Scalar)
-def _(value: pa.Scalar):
-    return value_to_sql(value.as_py())
-
-
@value_to_sql.register(np.ndarray)
 def _(value: np.ndarray):
    return value_to_sql(value.tolist())
@@ -2,11 +2,9 @@
 # SPDX-FileCopyrightText: Copyright The LanceDB Authors


-import inspect
 import re
 import sys
 from datetime import timedelta
-from importlib import resources
 import os
 from types import SimpleNamespace

@@ -19,10 +17,6 @@ from lance_namespace.errors import NamespaceNotEmptyError, TableNotFoundError
 from lancedb.pydantic import LanceModel, Vector


-def test_package_includes_pep_561_marker():
-    assert resources.files(lancedb).joinpath("py.typed").is_file()
-
-
 def test_basic(tmp_path):
    db = lancedb.connect(tmp_path)

@@ -68,44 +62,21 @@ def test_basic(tmp_path):
    assert db.open_table("test").name == db["test"].name


-def test_sync_debugger_inspection_does_not_use_background_loop(tmp_path, monkeypatch):
+def test_sync_repr_does_not_use_background_loop(tmp_path, monkeypatch):
    from lancedb.background_loop import LOOP

    db = lancedb.connect(tmp_path)
    table = db.create_table("test", data=[{"id": 1}])

    def fail_run(*args, **kwargs):
-        raise AssertionError("debugger inspection should not use the background loop")
+        raise AssertionError("repr should not use the Python background loop")

    monkeypatch.setattr(LOOP, "run", fail_run)

-    # Debuggers enumerate and evaluate every exposed attribute when expanding a
-    # variable. This must remain safe while their breakpoint suspends LOOP's thread.
-    members = dict(inspect.getmembers(db))
-
-    assert members["uri"] == str(tmp_path)
-    assert members["read_consistency_interval"] is None
    assert repr(db) == f"LanceDBConnection(uri={str(tmp_path)!r})"
    assert repr(table) == f"LanceTable(name='test', _conn={db!r})"


-def test_read_consistency_interval_does_not_use_background_loop(tmp_path, monkeypatch):
-    from lancedb.background_loop import LOOP
-    from lancedb.db import LanceDBConnection
-
-    consistency_interval = timedelta(seconds=5)
-    db = lancedb.connect(tmp_path, read_consistency_interval=consistency_interval)
-    db_from_inner = LanceDBConnection.from_inner(db._inner, consistency_interval)
-
-    def fail_run(*args, **kwargs):
-        raise AssertionError("properties should not use the Python background loop")
-
-    monkeypatch.setattr(LOOP, "run", fail_run)
-
-    assert db.read_consistency_interval == consistency_interval
-    assert db_from_inner.read_consistency_interval == consistency_interval
-
-
 def test_ingest_pd(tmp_path):
    db = lancedb.connect(tmp_path)

@@ -64,23 +64,6 @@ def test_embedding_function(tmp_path):
    assert np.allclose(actual, expected)


-def test_instructor_ndims_uses_instruction():
-    instructor = get_registry().get("instructor").create()
-    model = MagicMock()
-    model.encode.return_value = np.zeros((1, 384))
-
-    with patch.object(type(instructor), "get_model", return_value=model):
-        assert instructor.ndims() == 384
-
-    model.encode.assert_called_once_with(
-        [[instructor.source_instruction, "foo"]],
-        batch_size=instructor.batch_size,
-        show_progress_bar=instructor.show_progress_bar,
-        normalize_embeddings=instructor.normalize_embeddings,
-        device=instructor.device,
-    )
-
-
 def test_embedding_function_variables():
    @register("variable-testing")
    class VariableTestingFunction(TextEmbeddingFunction):
@@ -132,16 +115,34 @@ def test_embedding_function_variables():
    assert func.safe_model_dump()["secret_key"] == "$var:secret"


-def test_openai_variables_survive_metadata_round_trip():
+def test_parse_functions_with_variables():
+    @register("variable-parsing-test")
+    class VariableParsingFunction(TextEmbeddingFunction):
+        api_key: str
+        base_url: Optional[str] = None
+
+        @staticmethod
+        def sensitive_keys():
+            return ["api_key"]
+
+        def ndims(self):
+            return 10
+
+        def generate_embeddings(self, texts):
+            # Mock implementation that just returns random embeddings
+            # In real usage, this would use the api_key to call an API
+            return [np.random.rand(self.ndims()).tolist() for _ in texts]
+
    registry = EmbeddingFunctionRegistry.get_instance()

    registry.set_var("test_api_key", "sk-test-key-12345")
+    registry.set_var("test_base_url", "https://api.example.com")

    conf = EmbeddingFunctionConfig(
        source_column="text",
        vector_column="vector",
-        function=registry.get("openai").create(
-            api_key="$var:test_api_key", base_url="https://api.example.com"
+        function=registry.get("variable-parsing-test").create(
+            api_key="$var:test_api_key", base_url="$var:test_base_url"
        ),
    )

@@ -149,10 +150,7 @@ def test_openai_variables_survive_metadata_round_trip():

    # Create a mock arrow table with the metadata
    schema = pa.schema(
-        [
-            pa.field("text", pa.string()),
-            pa.field("vector", pa.list_(pa.float32(), 1536)),
-        ]
+        [pa.field("text", pa.string()), pa.field("vector", pa.list_(pa.float32(), 10))]
    )
    table = pa.table({"text": [], "vector": []}, schema=schema)
    table = table.replace_schema_metadata(metadata)
@@ -166,15 +164,13 @@ def test_openai_variables_survive_metadata_round_trip():

    assert parsed_func.api_key == "sk-test-key-12345"
    assert parsed_func.base_url == "https://api.example.com"
+
+    embeddings = parsed_func.generate_embeddings(["test text"])
+    assert len(embeddings) == 1
+    assert len(embeddings[0]) == 10
+
    assert parsed_func.safe_model_dump()["api_key"] == "$var:test_api_key"

-    with patch("lancedb.embeddings.openai.attempt_import_or_raise") as import_openai:
-        parsed_func._openai_client
-
-    import_openai.return_value.OpenAI.assert_called_once_with(
-        api_key="sk-test-key-12345", base_url="https://api.example.com"
-    )
-

 def test_embedding_with_bad_results(tmp_path):
    @register("null-embedding")
@@ -12,7 +12,7 @@ import pyarrow.compute as pc
 import pytest
 import pytest_asyncio

-from lancedb.index import BTree, FTS, IvfPq
+from lancedb.index import FTS
 from lancedb.table import AsyncTable, Table


@@ -99,86 +99,6 @@ async def test_async_hybrid_query_filters(table: AsyncTable):
    assert result["text"].to_pylist() == ["cat", "b"]


-@pytest.mark.asyncio
-async def test_hybrid_query_with_stale_fixed_size_binary_prefilter(
-    tmpdir_factory,
-):
-    tmp_path = str(tmpdir_factory.mktemp("stale_scalar_prefilter"))
-    db = await lancedb.connect_async(tmp_path)
-
-    def fixed_size_binary(value: int) -> bytes:
-        return value.to_bytes(16, byteorder="big")
-
-    num_rows = 1000
-    data = pa.table(
-        {
-            "space_id": pa.array(
-                [fixed_size_binary(i) for i in range(num_rows)],
-                type=pa.binary(16),
-            ),
-            "text": ["book"] * num_rows,
-            "vector": pa.array(
-                [[float(i), float(i)] for i in range(num_rows)],
-                type=pa.list_(pa.float32(), 2),
-            ),
-        }
-    )
-    table = await db.create_table("test", data)
-    await table.create_index(
-        "vector", config=IvfPq(num_partitions=4, num_sub_vectors=2)
-    )
-    await table.create_index("space_id", config=BTree())
-    await table.create_index("text", config=FTS(with_position=False))
-
-    # Advance the search indices without advancing the scalar index. This is the
-    # state that previously let hybrid search use an incomplete scalar prefilter.
-    await table.add(data)
-    lance_dataset = await table.to_lance()
-    lance_dataset.optimize.optimize_indices(index_names=["vector_idx", "text_idx"])
-    await table.checkout_latest()
-
-    scalar_stats = await table.index_stats("space_id_idx")
-    assert scalar_stats is not None
-    assert scalar_stats.num_indexed_rows == num_rows
-    assert scalar_stats.num_unindexed_rows == num_rows
-
-    for index_name in ["vector_idx", "text_idx"]:
-        search_stats = await table.index_stats(index_name)
-        assert search_stats is not None
-        assert search_stats.num_indexed_rows == num_rows * 2
-        assert search_stats.num_unindexed_rows == 0
-
-    matching_ids = [5, 10, 15, 20, 25, 30]
-    literals = [
-        f"arrow_cast(0x{fixed_size_binary(i).hex()}, 'FixedSizeBinary(16)')"
-        for i in matching_ids
-    ]
-    predicate = f"space_id IN ({', '.join(literals)})"
-    expected_ids = sorted(fixed_size_binary(i) for i in matching_ids for _ in range(2))
-
-    vector_query = (
-        table.query().where(predicate).nearest_to([5.0, 5.0]).limit(num_rows * 2)
-    )
-    vector_results = await vector_query.to_arrow()
-    assert sorted(vector_results["space_id"].to_pylist()) == expected_ids
-
-    fts_query = (
-        table.query().where(predicate).nearest_to_text("book").limit(num_rows * 2)
-    )
-    fts_results = await fts_query.to_arrow()
-    assert sorted(fts_results["space_id"].to_pylist()) == expected_ids
-
-    hybrid_results = await (
-        table.query()
-        .where(predicate)
-        .nearest_to([5.0, 5.0])
-        .nearest_to_text("book")
-        .limit(num_rows * 2)
-        .to_arrow()
-    )
-    assert sorted(hybrid_results["space_id"].to_pylist()) == expected_ids
-
-
@pytest.mark.asyncio
 async def test_async_hybrid_query_default_limit(table: AsyncTable):
    # add 10 new rows
@@ -203,19 +123,6 @@ async def test_async_hybrid_query_default_limit(table: AsyncTable):
    assert texts.count("a") == 1


-def test_hybrid_query_minimum_nprobes_zero_raises(sync_table: Table):
-    # minimum_nprobes(0) must raise the same validation error a plain vector
-    # query raises, not silently no-op because 0 is falsy.
-    with pytest.raises(ValueError, match="minimum_nprobes must be greater than 0"):
-        (
-            sync_table.search(query_type="hybrid")
-            .vector([0.0, 0.4])
-            .text("dog")
-            .minimum_nprobes(0)
-            .to_arrow()
-        )
-
-
 def test_hybrid_query_distance_range(sync_table: Table):
    reranker = RRFReranker(return_score="all")
    result = (
@@ -1,33 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright The LanceDB Authors
-
-import re
-import shutil
-import subprocess
-import sys
-
-import lancedb._lancedb as _lancedb
-import pytest
-
-
-@pytest.mark.skipif(sys.platform != "linux", reason="ldd is Linux-specific")
-def test_native_extension_does_not_link_openssl():
-    """OpenSSL-linked wheels abort when imported on RHEL hosts in FIPS mode."""
-    ldd = shutil.which("ldd")
-    if ldd is None:
-        pytest.skip("ldd is not installed")
-
-    result = subprocess.run(
-        [ldd, _lancedb.__file__],
-        check=True,
-        capture_output=True,
-        text=True,
-    )
-    openssl_libraries = re.findall(
-        r"^\s*(lib(?:crypto|ssl)\S*)\s+=>", result.stdout, flags=re.MULTILINE
-    )
-
-    assert not openssl_libraries, (
-        "the LanceDB native extension must use rustls instead of linking OpenSSL: "
-        f"{openssl_libraries}"
-    )
@@ -372,31 +372,6 @@ async def test_create_vector_index(some_table: AsyncTable):
    assert stats.num_indices == 1


-@pytest.mark.asyncio
-async def test_create_ivf_index_reports_unsplittable_partitions(db_async):
-    dim = 8
-    num_partitions = 300  # More than 256 selects hierarchical k-means.
-    base_vectors = [[float(row == column) for column in range(dim)] for row in range(5)]
-    vectors = pa.array(base_vectors * 200, pa.list_(pa.float32(), dim))
-    table = await db_async.create_table(
-        "unsplittable_partitions",
-        pa.table({"vector": vectors}),
-    )
-
-    error_pattern = (
-        rf"Cannot create {num_partitions} IVF partitions: k-means could only form"
-    )
-    with pytest.raises(RuntimeError, match=error_pattern):
-        await table.create_index(
-            "vector",
-            config=IvfFlat(
-                distance_type="dot",
-                num_partitions=num_partitions,
-                max_iterations=10,
-            ),
-        )
-
-
@pytest.mark.asyncio
 async def test_create_4bit_ivfpq_index(some_table: AsyncTable):
    # Can create
@@ -1,42 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright The LanceDB Authors
-
-import importlib
-import re
-import sys
-from pathlib import Path
-
-import pytest
-
-
-def test_pyo3_abi_matches_minimum_supported_python():
-    project_dir = Path(__file__).parents[2]
-    pyproject = (project_dir / "pyproject.toml").read_text()
-    cargo_manifest = (project_dir / "Cargo.toml").read_text()
-
-    minimum_python = re.search(
-        r'^requires-python\s*=\s*">=(\d+)\.(\d+)"$', pyproject, re.MULTILINE
-    )
-    assert minimum_python is not None
-
-    major, minor = minimum_python.groups()
-    expected_abi = f"abi3-py{major}{minor}"
-    configured_abis = re.findall(r'"(abi3-py\d+)"', cargo_manifest)
-
-    assert configured_abis == [expected_abi, expected_abi], (
-        "the pyo3 runtime and build ABI features must both match requires-python"
-    )
-
-
-@pytest.mark.skipif(sys.platform != "win32", reason="Windows wheel regression test")
-def test_windows_wheel_tag_and_native_import():
-    project_dir = Path(__file__).parents[2]
-    wheels = list((project_dir.parent / "target" / "wheels").glob("lancedb-*.whl"))
-    if not wheels:
-        pytest.skip("no wheel artifact is available in this development environment")
-
-    assert len(wheels) == 1
-    assert wheels[0].name.endswith("-cp310-abi3-win_amd64.whl")
-
-    native_module = importlib.import_module("lancedb._lancedb")
-    assert Path(native_module.__file__).suffix == ".pyd"
@@ -6,7 +6,6 @@ import math
 import pytest

 from lancedb import DBConnection, Table, connect
-from lancedb.background_loop import LOOP
 from lancedb.permutation import Permutation, Permutations, permutation_builder


@@ -32,25 +31,6 @@ def test_split_random_ratios(mem_db):
    assert 65 <= split_1_count <= 75  # ~70% ± tolerance


-def test_execute_does_not_reenter_background_loop(tmp_path, monkeypatch):
-    import threading
-
-    db = connect(tmp_path)
-    tbl = db.create_table("test_table", pa.table({"x": range(10)}))
-    original_run = LOOP.run
-
-    def fail_on_reentry(future):
-        assert threading.current_thread() is not LOOP.thread
-        return original_run(future)
-
-    monkeypatch.setattr(LOOP, "run", fail_on_reentry)
-
-    permutation_tbl = permutation_builder(tbl).execute()
-
-    assert permutation_tbl.count_rows() == 10
-    assert permutation_tbl._conn.read_consistency_interval is None
-
-
 def test_split_random_counts(mem_db):
    """Test random splitting with absolute counts."""
    tbl = mem_db.create_table(
@@ -9,7 +9,6 @@ from typing import List, Optional, Tuple
 import pyarrow as pa
 import pydantic
 import pytest
-from lancedb.conftest import MockTextEmbeddingFunction
 from lancedb.pydantic import (
    PYDANTIC_VERSION,
    LanceModel,
@@ -416,36 +415,6 @@ def test_nullable_vector():
    assert schema == pa.schema([pa.field("vec", pa.list_(pa.float32(), 16), True)])


-def test_bare_vector_raises_clear_error():
-    namespace = {
-        "__name__": "test_model_without_pyarrow",
-        "LanceModel": LanceModel,
-        "Vector": Vector,
-    }
-
-    with pytest.raises(TypeError, match=r"Vector must be parameterized.*Vector\(128\)"):
-        exec("class TestModel(LanceModel):\n    vector: Vector", namespace)
-
-
-def test_embedding_vector_list_annotation():
-    embedding = MockTextEmbeddingFunction.create()
-
-    class StaticTypingModel(LanceModel):
-        text: str = embedding.SourceField()
-        vector: list[float] = embedding.VectorField()
-
-    schema = pydantic_to_schema(StaticTypingModel)
-    assert schema == pa.schema(
-        [
-            pa.field("text", pa.utf8(), False),
-            pa.field("vector", pa.list_(pa.float32(), embedding.ndims()), True),
-        ]
-    )
-
-    model = StaticTypingModel(text="hello", vector=[0.0] * embedding.ndims())
-    assert model.vector == [0.0] * embedding.ndims()
-
-
 def test_fixed_size_list_field():
    class TestModel(pydantic.BaseModel):
        vec: Vector(16)
@@ -570,15 +570,6 @@ def test_query_builder(table):
    assert all(np.array(rs[0]["vector"]) == [1, 2])


-def test_query_multiple_vectors(table):
-    results = table.search([np.array([1, 2]), np.array([4, 5])]).limit(1).to_list()
-
-    assert len(results) == 2
-    results_by_query = {result["query_index"]: result for result in results}
-    assert results_by_query[0]["id"] == 1
-    assert results_by_query[1]["id"] == 2
-
-
 def test_with_row_id(table: lancedb.table.Table):
    rs = table.search().with_row_id(True).to_arrow()
    assert "_rowid" in rs.column_names
@@ -35,12 +35,6 @@ def make_mock_http_handler(handler):
    return MockLanceDBHandler


-@pytest.mark.parametrize("db_name", ["a" * 64, "invalid..database"])
-def test_connect_rejects_invalid_cloud_dns_hostname(db_name):
-    with pytest.raises(ValueError, match="DNS labels must contain 1 to 63 bytes"):
-        lancedb.connect(f"db://{db_name}", api_key="fake")
-
-
@contextlib.contextmanager
 def mock_lancedb_connection(handler):
    with http.server.HTTPServer(
@@ -2061,24 +2055,6 @@ def blob_remote_table(*, server_version=Version("0.5.0")):
            request.send_header("phalanx-version", str(server_version))
            request.end_headers()
            request.wfile.write(json.dumps(BLOB_DESCRIBE_RESPONSE).encode())
-        elif request.path.startswith("/v1/table/test/blob/image/"):
-            path = request.path.partition("?")[0]
-            row_id = int(path.split("/")[-2])
-            payload = {10: b"alpha", 20: None, 30: b"gamma"}[row_id]
-            if payload is None:
-                request.send_response(204)
-                request.end_headers()
-                return
-            byte_range = request.headers["Range"].removeprefix("bytes=")
-            start_text, end_text = byte_range.split("-", maxsplit=1)
-            start = int(start_text)
-            end = int(end_text) if end_text else len(payload) - 1
-            chunk = payload[start : end + 1]
-            request.send_response(206)
-            request.send_header("Content-Range", f"bytes {start}-{end}/{len(payload)}")
-            request.send_header("Content-Length", str(len(chunk)))
-            request.end_headers()
-            request.wfile.write(chunk)
        elif request.path == "/v1/table/test/query/":
            content_len = int(request.headers.get("Content-Length", 0))
            body = json.loads(request.rfile.read(content_len))
@@ -2116,21 +2092,8 @@ def test_remote_blob_columns_and_fetch():
        assert table.blob_columns() == ["image"]
        blobs = table.fetch_blobs("image", [10, 20, 30])
        assert blobs.to_pylist() == [b"alpha", None, b"gamma"]
-
-
-def test_remote_blob_files_are_lazy_seekable_handles():
-    with blob_remote_table() as table:
-        files = table.fetch_blob_files("image", [10, 20, 30])
-
-        assert len(files) == 3
-        alpha, null_row, gamma = files
-        assert null_row is None
-        assert alpha is not None
-        assert gamma is not None
-        assert alpha.size() == 5
-        assert alpha.read_range(1, 3) == b"lph"
-        gamma.seek(2)
-        assert gamma.read() == b"mma"
+        with pytest.raises(NotImplementedError, match="Use fetch_blobs for full bytes"):
+            table.fetch_blob_files("image", [10, 20, 30])


 def test_remote_blob_fetch_accepts_query_table():
@@ -2,14 +2,10 @@
 # SPDX-FileCopyrightText: Copyright The LanceDB Authors


-import ctypes
-import gc
 import os
 import sys
 import threading
 import warnings
-import weakref
-from concurrent.futures import ThreadPoolExecutor
 from datetime import date, datetime, timedelta
 from time import sleep
 from typing import List
@@ -102,30 +98,6 @@ def test_basic(mem_db: DBConnection):
    assert table.to_arrow() == expected_data


-def test_search_preserves_nulls_from_sliced_arrow_table(mem_db: DBConnection):
-    data = pa.table(
-        {
-            "id": [0, 1, 2, 3, 4],
-            "score_cn": [None, 22, None, 5, 8],
-            "score_mt": [None, 42, None, 5, 8],
-            "vector": [
-                [20, 19, -1, -1],
-                [41, 38, 22, 42],
-                [10, 10, -1, -1],
-                [5, 5, 5, 5],
-                [8, 8, 8, 8],
-            ],
-        }
-    ).slice(1)
-
-    table = mem_db.create_table("sliced_nullable", data=data)
-    result = table.search([41, 38, 22, 42]).limit(1).to_arrow()
-
-    assert result["id"].to_pylist() == [1]
-    assert result["score_cn"].to_pylist() == [22]
-    assert result["score_mt"].to_pylist() == [42]
-
-
 def test_table_to_pandas_default_matches_arrow(tmp_db: DBConnection):
    pd = pytest.importorskip("pandas")
    data = pa.table({"id": [1, 2], "text": ["one", "two"]})
@@ -462,38 +434,6 @@ def test_add(mem_db: DBConnection):
    _add(table, schema)


-def test_add_releases_arrow_buffers_without_gc(mem_db: DBConnection):
-    """Regression test for https://github.com/lancedb/lancedb/issues/2512."""
-    schema = pa.schema([pa.field("x", pa.int64())])
-    table = mem_db.create_table("test_add_releases_arrow_buffers", schema=schema)
-
-    class BufferOwner:
-        def __init__(self, size: int):
-            self.memory = ctypes.create_string_buffer(size)
-
-    owner_refs = []
-    gc_was_enabled = gc.isenabled()
-    gc.disable()
-    try:
-        for _ in range(3):
-            size = 8 * 1024
-            owner = BufferOwner(size)
-            arrow_buffer = pa.foreign_buffer(
-                ctypes.addressof(owner.memory), size, owner
-            )
-            array = pa.Array.from_buffers(pa.int64(), 1024, [None, arrow_buffer])
-            batch = pa.RecordBatch.from_arrays([array], schema=schema)
-            owner_refs.append(weakref.ref(owner))
-
-            table.add(batch)
-            del batch, array, arrow_buffer, owner
-
-        assert all(owner_ref() is None for owner_ref in owner_refs)
-    finally:
-        if gc_was_enabled:
-            gc.enable()
-
-
 def test_add_write_parallelism(mem_db: DBConnection):
    schema = pa.schema([pa.field("id", pa.int64())])
    table = mem_db.create_table("test", schema=schema)
@@ -929,7 +869,6 @@ def test_polars(mem_db: DBConnection):

    # enter table to polars dataframe
    result = table.to_polars()
-    assert isinstance(result, pl.LazyFrame)
    assert np.allclose(result.collect()["vector"].to_list(), data["vector"])

    # make sure filtering isn't broken
@@ -1846,27 +1785,6 @@ def test_add_with_empty_fixed_size_list_drops_bad_rows(mem_db: DBConnection):
    assert np.allclose(data["embedding"].to_pylist()[0], np.array([0.1] * 16))


-def test_add_nullable_fixed_size_list_with_none(mem_db: DBConnection):
-    """Regression test for issue #2340."""
-    table = mem_db.create_table(
-        "test_nullable_fixed_size_list",
-        schema=pa.schema(
-            [
-                pa.field("id", pa.string()),
-                pa.field("feature", pa.list_(pa.float32(), 256)),
-                pa.field("tags", pa.list_(pa.string())),
-            ]
-        ),
-    )
-
-    table.add([{"id": "1", "feature": None, "tags": ["tag1", "tag2"]}])
-
-    result = table.to_arrow()
-    assert result.to_pylist() == [
-        {"id": "1", "feature": None, "tags": ["tag1", "tag2"]}
-    ]
-
-
 def test_add_nullable_struct_with_none(mem_db: DBConnection):
    """Regression test for issue #2654: a nullable struct column whose
    first batch contains only None values must not crash in
@@ -1906,33 +1824,6 @@ def test_add_nullable_struct_with_none(mem_db: DBConnection):
    assert result.column("data").to_pylist() == [{"x": 1.0}, None]


-def test_read_mostly_null_list_v2_2_page_boundary(tmp_path):
-    # Regression test for #3194. This row/value count crosses a v2.2 structural
-    # encoding page boundary where Lance 3.0.0 sliced repetition/definition
-    # levels by row offset and decoded child arrays at different lengths.
-    num_rows = 64_885
-    num_values = 217
-    list_type = pa.list_(pa.float32())
-    source = pa.table(
-        {
-            "id": np.arange(num_rows, dtype=np.int64),
-            "coords": pa.array(
-                [[1.0, 2.0, 3.0, 4.0]] * num_values + [None] * (num_rows - num_values),
-                type=list_type,
-            ),
-        }
-    )
-    db = lancedb.connect(
-        tmp_path,
-        storage_options={"new_table_data_storage_version": "2.2"},
-    )
-    table = db.create_table("test_sparse_nullable_list", data=source)
-
-    result = table.search().select(["id", "coords"]).limit(num_rows).to_arrow()
-
-    assert result.equals(source)
-
-
 def test_add_with_integer_embeddings_preserves_casting(mem_db: DBConnection):
    class Schema(LanceModel):
        text: str
@@ -2218,45 +2109,6 @@ def test_merge(tmp_db: DBConnection, tmp_path):
    table.merge(other_dataset, left_on="id")


-@pytest.mark.parametrize("storage_version", ["legacy", "stable"])
-def test_search_after_merge(tmp_path, storage_version):
-    pytest.importorskip("lance")
-    pd = pytest.importorskip("pandas")
-
-    db = lancedb.connect(
-        tmp_path,
-        storage_options={"new_table_data_storage_version": storage_version},
-    )
-    rng = np.random.default_rng(42)
-    row_count = 512
-    vectors = rng.standard_normal((row_count, 8)).astype(np.float32)
-    table = db.create_table(
-        "search_after_merge",
-        data=pd.DataFrame(
-            {
-                "id": [str(i) for i in range(row_count)],
-                "vector": list(vectors),
-            }
-        ),
-    )
-    table.create_index("vector", config=IvfPq(num_partitions=1, num_sub_vectors=2))
-
-    links = pd.DataFrame(
-        {
-            "id": [str(i) for i in range(row_count // 2)],
-            "link": [f"https://example.com/{i}" for i in range(row_count // 2)],
-        }
-    )
-    table.merge(links, left_on="id")
-
-    query = table.search(vectors[-1]).refine_factor(50).limit(10)
-    assert "ANN" in query.explain_plan(verbose=True)
-
-    result = query.to_arrow()
-    links_by_id = dict(zip(result["id"].to_pylist(), result["link"].to_pylist()))
-    assert links_by_id[str(row_count - 1)] is None
-
-
 def test_delete(mem_db: DBConnection):
    table = mem_db.create_table(
        "my_table",
@@ -2272,27 +2124,6 @@ def test_delete(mem_db: DBConnection):
    assert table.to_arrow()["id"].to_pylist() == [1]


-def test_concurrent_deletes_are_thread_safe(mem_db: DBConnection):
-    num_workers = 8
-    table = mem_db.create_table(
-        "my_table", data=[{"id": row_id} for row_id in range(num_workers)]
-    )
-    barrier = threading.Barrier(num_workers)
-
-    def delete(row_id: int):
-        barrier.wait()
-        return table.delete(f"id = {row_id}")
-
-    with ThreadPoolExecutor(max_workers=num_workers) as pool:
-        results = list(pool.map(delete, range(num_workers)))
-
-    assert all(result.num_deleted_rows == 1 for result in results)
-    assert sorted(result.version for result in results) == list(
-        range(2, num_workers + 2)
-    )
-    assert table.count_rows() == 0
-
-
 def test_delete_expr(mem_db: DBConnection):
    table = mem_db.create_table(
        "my_table",
@@ -2343,20 +2174,6 @@ def test_update(mem_db: DBConnection):
    assert np.allclose(v, np.array([[1.2, 1.9], [1.1, 1.1]]))


-def test_update_with_arrow_scalar(mem_db: DBConnection):
-    schema = pa.schema({"id": pa.int64(), "vector": pa.list_(pa.float32(), 4)})
-    table = mem_db.create_table("my_table", schema=schema)
-    table.add([{"id": 1, "vector": [1.0, 2.0, 3.0, 4.0]}])
-
-    value = table.search().select(["vector"]).limit(1).to_arrow()["vector"][0]
-    assert isinstance(value, pa.FixedSizeListScalar)
-
-    result = table.update(where="id == 1", values={"vector": value})
-
-    assert result.rows_updated == 1
-    assert table.to_arrow()["vector"].to_pylist() == [[1.0, 2.0, 3.0, 4.0]]
-
-
 def test_update_types(mem_db: DBConnection):
    table = mem_db.create_table(
        "my_table",
@@ -2524,55 +2341,6 @@ def test_merge_insert(mem_db: DBConnection):
        )


-def test_merge_insert_nullable_pandas_into_pydantic_schema(mem_db: DBConnection):
-    # Regression test for https://github.com/lancedb/lancedb/issues/2366
-    pd = pytest.importorskip("pandas")
-
-    class Document(LanceModel):
-        id: int
-        title: str
-        content: str
-
-    table = mem_db.create_table("documents", schema=Document)
-    table.add(
-        pd.DataFrame(
-            {
-                "title": ["Old title", "Unchanged"],
-                "id": [2, 3],
-                "content": ["Old content", "Keep this"],
-            }
-        )
-    )
-
-    # Pandas produces nullable Arrow fields, in an order that differs from the
-    # non-nullable Pydantic schema. This is valid as long as the data has no nulls.
-    new_data = pd.DataFrame(
-        {
-            "title": ["Inserted", "Updated"],
-            "id": [1, 2],
-            "content": ["New row", "New content"],
-        }
-    )
-    result = (
-        table.merge_insert("id")
-        .when_matched_update_all()
-        .when_not_matched_insert_all()
-        .execute(new_data)
-    )
-
-    assert result.num_inserted_rows == 1
-    assert result.num_updated_rows == 1
-    expected = pa.Table.from_pylist(
-        [
-            {"id": 1, "title": "Inserted", "content": "New row"},
-            {"id": 2, "title": "Updated", "content": "New content"},
-            {"id": 3, "title": "Unchanged", "content": "Keep this"},
-        ],
-        schema=Document.to_arrow_schema(),
-    )
-    assert table.to_arrow().sort_by("id") == expected
-
-
 def test_merge_insert_by_source_delete_expr(mem_db: DBConnection):
    table = mem_db.create_table(
        "my_table",
@@ -2596,29 +2364,6 @@ def test_merge_insert_by_source_delete_expr(mem_db: DBConnection):
    assert table.to_arrow().sort_by("a") == expected


-def test_merge_insert_by_source_delete_reconfigure(mem_db: DBConnection):
-    # Calling when_not_matched_by_source_delete() again with no condition must
-    # widen the delete to unconditional, not keep the earlier condition around.
-    table = mem_db.create_table(
-        "my_table",
-        data=pa.table({"a": [1, 2, 3], "b": ["a", "b", "c"]}),
-    )
-    new_data = pa.table({"a": [2, 4], "b": ["x", "z"]})
-
-    merge_insert_res = (
-        table.merge_insert("a")
-        .when_matched_update_all()
-        .when_not_matched_insert_all()
-        .when_not_matched_by_source_delete("a > 2")
-        .when_not_matched_by_source_delete()
-        .execute(new_data)
-    )
-    assert merge_insert_res.num_deleted_rows == 2
-
-    expected = pa.table({"a": [2, 4], "b": ["x", "z"]})
-    assert table.to_arrow().sort_by("a") == expected
-
-
@pytest.mark.asyncio
 async def test_merge_insert_by_source_delete_expr_async(
    mem_db_async: AsyncConnection,
@@ -2673,36 +2418,6 @@ def test_merge_insert_subschema(mem_db: DBConnection, data_format):
    assert table.to_arrow().sort_by("id") == expected


-def test_repeated_partial_merge_insert_with_scalar_index(mem_db: DBConnection):
-    def make_batch(start: int) -> pa.Table:
-        return pa.table(
-            {
-                "id": [f"id-{i:04}" for i in range(start, start + 100)],
-                "category": ["A"] * 100,
-                "value_a": [float(i) for i in range(start, start + 100)],
-                "value_b": [float(i) / 10 for i in range(100)],
-            }
-        )
-
-    table = mem_db.create_table("my_table", data=make_batch(0))
-    table.add(make_batch(100))
-    table.add(make_batch(200))
-    table.create_index("id", config=BTree())
-
-    ids = [f"id-{i:04}" for i in range(100, 200)]
-    for value in (999.0, 888.0):
-        result = (
-            table.merge_insert("id")
-            .when_matched_update_all()
-            .execute(pa.table({"id": ids, "value_a": [value] * 100}))
-        )
-        assert result.num_updated_rows == 100
-
-    actual = table.to_arrow().sort_by("id")
-    assert actual.num_rows == 300
-    assert actual["value_a"].to_pylist()[100:200] == [888.0] * 100
-
-
@pytest.mark.asyncio
 async def test_merge_insert_async(mem_db_async: AsyncConnection):
    data = pa.table({"a": [1, 2, 3], "b": ["a", "b", "c"]})
@@ -2799,40 +2514,15 @@ def test_create_with_embedding_function(mem_db: DBConnection):
    assert actual == expected


-def test_create_f16_table_from_arrow_data(mem_db: DBConnection):
-    dimension = 32
-    num_rows = 512
-    values = pa.array(
-        np.random.default_rng(42)
-        .standard_normal(num_rows * dimension)
-        .astype(np.float16)
-    )
-    df = pa.table(
-        {
-            "text": [f"s-{i}" for i in range(num_rows)],
-            "vector": pa.FixedSizeListArray.from_arrays(values, dimension),
-        }
-    )
-    table = mem_db.create_table("f16_tbl", data=df)
-    assert table.schema.field("vector").type == pa.list_(pa.float16(), dimension)
-    table.create_index(num_partitions=2, num_sub_vectors=2)
-
-    query = df["vector"][2].as_py()
-    expected = table.search(query).limit(2).to_arrow()
-
-    assert "s-2" in expected["text"].to_pylist()
-
-
 def test_create_f16_table(mem_db: DBConnection):
    class MyTable(LanceModel):
        text: str
        vector: Vector(32, value_type=pa.float16())

-    rng = np.random.default_rng(42)
    df = pa.table(
        {
            "text": [f"s-{i}" for i in range(512)],
-            "vector": [rng.standard_normal(32).astype(np.float16) for _ in range(512)],
+            "vector": [np.random.randn(32).astype(np.float16) for _ in range(512)],
        }
    )
    table = mem_db.create_table(
@@ -3754,8 +3444,8 @@ def test_create_table_empty_list_no_schema_error(mem_db: DBConnection):
        mem_db.create_table("test_empty_no_schema", data=[])


-def test_create_table_without_data_with_vector_schema(tmp_path):
-    """Test exact scenario from issue #1968.
+def test_add_table_with_empty_embeddings(tmp_path):
+    """Test exact scenario from issue #1968

    Regression test for issue #1968:
    https://github.com/lancedb/lancedb/issues/1968
@@ -3767,9 +3457,6 @@ def test_create_table_without_data_with_vector_schema(tmp_path):
        embedding: Vector(16)

    table = db.create_table("test", schema=MySchema)
-    assert table.count_rows() == 0
-    assert table.schema == MySchema.to_arrow_schema()
-
    table.add(
        [{"text": "bar", "embedding": [0.1] * 16}],
        on_bad_vectors="drop",
@@ -75,22 +75,6 @@ class TestVoyageAIModelRegistration:
        with pytest.raises(ValueError, match="not supported"):
            func.ndims()

-    def test_voyage3_source_embeddings_use_text_api(self, mock_voyageai_client):
-        """Regression test for text table data being sent to the multimodal API."""
-        mock_voyageai_client.tokenize.return_value = [["hello", "world"]]
-        mock_voyageai_client.embed.return_value.embeddings = [[0.1] * 1024]
-
-        registry = get_registry()
-        func = registry.get("voyageai").create(name="voyage-3")
-
-        embeddings = func.compute_source_embeddings("hello world")
-
-        assert embeddings == [[0.1] * 1024]
-        mock_voyageai_client.embed.assert_called_once_with(
-            texts=["hello world"], model="voyage-3", input_type="document"
-        )
-        mock_voyageai_client.multimodal_embed.assert_not_called()
-
    @pytest.mark.parametrize(
        "model_name",
        [
@@ -1,15 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-# SPDX-FileCopyrightText: Copyright The LanceDB Authors
-
-from typing import assert_type
-
-import lancedb
-from lancedb import AsyncConnection, DBConnection
-
-
-def check_connect_type() -> None:
-    assert_type(lancedb.connect("memory://"), DBConnection)
-
-
-async def check_connect_async_type() -> None:
-    assert_type(await lancedb.connect_async("memory://"), AsyncConnection)
@@ -426,11 +426,9 @@ pub struct PyBlobFile {
 impl PyBlobFile {
    fn read_bytes(self_: PyRef<'_, Self>) -> PyResult<Py<PyBytes>> {
        let inner = self_.inner.clone();
-        let py = self_.py();
-        let bytes = py
-            .detach(move || block_on(async move { inner.read().await }))
+        let bytes = block_on(async move { inner.read().await })
            .map_err(|e| PyRuntimeError::new_err(format!("blob read failed: {e}")))?;
-        Ok(PyBytes::new(py, bytes.as_ref()).unbind())
+        Ok(PyBytes::new(self_.py(), bytes.as_ref()).unbind())
    }

    pub fn read(self_: PyRef<'_, Self>) -> PyResult<Bound<'_, PyAny>> {
@@ -446,32 +444,24 @@ impl PyBlobFile {

    fn close(self_: PyRef<'_, Self>) -> PyResult<()> {
        let inner = self_.inner.clone();
-        self_
-            .py()
-            .detach(move || block_on(async move { inner.close().await }))
+        block_on(async move { inner.close().await })
            .map_err(|e| PyRuntimeError::new_err(format!("blob close failed: {e}")))
    }

    fn is_closed(self_: PyRef<'_, Self>) -> bool {
        let inner = self_.inner.clone();
-        self_
-            .py()
-            .detach(move || block_on(async move { inner.is_closed().await }))
+        block_on(async move { inner.is_closed().await })
    }

    fn seek(self_: PyRef<'_, Self>, position: u64) -> PyResult<()> {
        let inner = self_.inner.clone();
-        self_
-            .py()
-            .detach(move || block_on(async move { inner.seek(position).await }))
+        block_on(async move { inner.seek(position).await })
            .map_err(|e| PyRuntimeError::new_err(format!("blob seek failed: {e}")))
    }

    fn tell(self_: PyRef<'_, Self>) -> PyResult<u64> {
        let inner = self_.inner.clone();
-        self_
-            .py()
-            .detach(move || block_on(async move { inner.tell().await }))
+        block_on(async move { inner.tell().await })
            .map_err(|e| PyRuntimeError::new_err(format!("blob tell failed: {e}")))
    }

@@ -485,20 +475,16 @@ impl PyBlobFile {
            .checked_add(length as u64)
            .ok_or_else(|| PyValueError::new_err("offset + length overflowed"))?;
        let inner = self_.inner.clone();
-        let py = self_.py();
-        let bytes = py
-            .detach(move || block_on(async move { inner.read_range(offset..end).await }))
+        let bytes = block_on(async move { inner.read_range(offset..end).await })
            .map_err(|e| PyRuntimeError::new_err(format!("blob read_range failed: {e}")))?;
-        Ok(PyBytes::new(py, bytes.as_ref()).unbind())
+        Ok(PyBytes::new(self_.py(), bytes.as_ref()).unbind())
    }

    fn read_up_to(self_: PyRef<'_, Self>, length: usize) -> PyResult<Py<PyBytes>> {
        let inner = self_.inner.clone();
-        let py = self_.py();
-        let bytes = py
-            .detach(move || block_on(async move { inner.read_up_to(length).await }))
-            .map_err(|e| PyRuntimeError::new_err(format!("blob read_up_to failed: {e}")))?;
-        Ok(PyBytes::new(py, bytes.as_ref()).unbind())
+        let bytes = block_on(async move { inner.read_up_to(length).await })
+            .map_err(|e| PyRuntimeError::new_err(format!("blob read failed: {e}")))?;
+        Ok(PyBytes::new(self_.py(), bytes.as_ref()).unbind())
    }
 }

@@ -745,9 +731,6 @@ impl Table {

    #[allow(private_interfaces)]
    pub fn delete(self_: PyRef<'_, Self>, condition: PredicateArg) -> PyResult<Bound<'_, PyAny>> {
-        // Do not hold the Python borrow across the await.  The cloned Rust table
-        // handle is thread-safe and allows deletes on the same Python table to
-        // run concurrently without PyO3 reporting "Already borrowed".
        let inner = self_.inner_ref()?.clone();
        future_into_py(self_.py(), async move {
            let result = match &condition {
@@ -1378,12 +1361,7 @@ impl Table {

        let inner = self_.inner_ref()?.clone();
        future_into_py(self_.py(), async move {
-            let result = inner
-                .add_columns()
-                .transform(definitions)
-                .execute()
-                .await
-                .infer_error()?;
+            let result = inner.add_columns(definitions, None).await.infer_error()?;
            Ok(AddColumnsResult::from(result))
        })
    }
@@ -1397,12 +1375,7 @@ impl Table {

        let inner = self_.inner_ref()?.clone();
        future_into_py(self_.py(), async move {
-            let result = inner
-                .add_columns()
-                .transform(transform)
-                .execute()
-                .await
-                .infer_error()?;
+            let result = inner.add_columns(transform, None).await.infer_error()?;
            Ok(AddColumnsResult::from(result))
        })
    }
@@ -1998,7 +1998,7 @@ requires-dist = [
    { name = "pillow", marker = "extra == 'clip'", specifier = ">=12.1.1" },
    { name = "pillow", marker = "extra == 'embeddings'", specifier = ">=12.1.1" },
    { name = "pillow", marker = "extra == 'siglip'", specifier = ">=12.1.1" },
-    { name = "polars", marker = "extra == 'tests'", specifier = ">=0.19,<=1.32.3" },
+    { name = "polars", marker = "extra == 'tests'", specifier = ">=0.19,<=1.3.0" },
    { name = "pre-commit", marker = "extra == 'dev'", specifier = ">=3.5.0" },
    { name = "pyarrow", specifier = ">=16" },
    { name = "pyarrow", marker = "extra == 'tests'", specifier = "<25" },
@@ -49,8 +49,8 @@ lance-namespace = { workspace = true }
 lance-namespace-impls = { workspace = true }
 metrics = { workspace = true, optional = true }
 metrics-util = { workspace = true, optional = true }
-# Pin the GooseFS SDK to the version required by Lance's OpenDAL dependency.
-goosefs-sdk = { version = "=0.1.9", optional = true }
+# Keep the direct dependency aligned with the version required by OpenDAL.
+goosefs-sdk = { version = "=0.1.8", optional = true }
 moka = { workspace = true }
 pin-project = { workspace = true }
 tokio = { version = "1.23", features = ["rt-multi-thread", "sync"] }
@@ -75,8 +75,6 @@ reqwest = { version = "0.12.0", default-features = false, features = [
    "http2",
    "json",
    "macos-system-configuration",
-    # Avoid linking OpenSSL into Python wheels, which breaks on FIPS hosts.
-    "rustls-tls-native-roots",
    "stream",
 ], optional = true }
 http = { version = "1", optional = true } # Matching what is in reqwest
@@ -9,7 +9,6 @@
 //!
 //! Blob tables require Lance file format >= 2.2 and stable row ids at create.

-use std::ops::Range;
 use std::sync::Arc;

 use arrow_array::LargeBinaryArray;
@@ -17,203 +16,11 @@ use arrow_array::builder::LargeBinaryBuilder;
 use arrow_schema::{DataType, Field, Schema};
 use lance::dataset::{BlobRangeRequest as LanceBlobRangeRequest, Dataset, WriteParams};
 use lance_arrow::FieldExt;
-use lance_file::version::LanceFileVersion;
-use lance_io::object_store::ObjectStore;
-use object_store::path::Path;
+use lance_encoding::version::LanceFileVersion;

 use crate::error::{Error, Result};

-/// Seekable handle for one blob value, backed by local storage or a remote
-/// HTTP byte-range endpoint.
-#[derive(Debug)]
-pub struct BlobFile {
-    inner: BlobFileInner,
-}
-
-#[derive(Debug)]
-enum BlobFileInner {
-    Native(lance::dataset::BlobFile),
-    #[cfg(feature = "remote")]
-    Remote(Box<crate::remote::table::blobs::RemoteBlobFile>),
-}
-
-impl From<lance::dataset::BlobFile> for BlobFile {
-    fn from(value: lance::dataset::BlobFile) -> Self {
-        Self {
-            inner: BlobFileInner::Native(value),
-        }
-    }
-}
-
-#[cfg(feature = "remote")]
-impl From<crate::remote::table::blobs::RemoteBlobFile> for BlobFile {
-    fn from(value: crate::remote::table::blobs::RemoteBlobFile) -> Self {
-        Self {
-            inner: BlobFileInner::Remote(Box::new(value)),
-        }
-    }
-}
-
-impl BlobFile {
-    /// Inline reader over a data-file slice.
-    pub fn new_inline(
-        object_store: Arc<ObjectStore>,
-        path: Path,
-        position: u64,
-        size: u64,
-    ) -> Self {
-        lance::dataset::BlobFile::new_inline(object_store, path, position, size).into()
-    }
-
-    /// Dedicated sidecar-file reader.
-    pub fn new_dedicated(object_store: Arc<ObjectStore>, path: Path, size: u64) -> Self {
-        lance::dataset::BlobFile::new_dedicated(object_store, path, size).into()
-    }
-
-    /// Packed reader for a slice in a shared sidecar.
-    pub fn new_packed(
-        object_store: Arc<ObjectStore>,
-        path: Path,
-        position: u64,
-        size: u64,
-    ) -> Self {
-        lance::dataset::BlobFile::new_packed(object_store, path, position, size).into()
-    }
-
-    /// External reader at a resolved object location.
-    pub fn new_external(
-        object_store: Arc<ObjectStore>,
-        path: Path,
-        uri: String,
-        position: u64,
-        size: u64,
-    ) -> Self {
-        lance::dataset::BlobFile::new_external(object_store, path, uri, position, size).into()
-    }
-
-    /// Close the handle.
-    pub async fn close(&self) -> lance_core::Result<()> {
-        match &self.inner {
-            BlobFileInner::Native(file) => file.close().await,
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(file) => file.close().await,
-        }
-    }
-
-    /// Whether the handle is closed.
-    pub async fn is_closed(&self) -> bool {
-        match &self.inner {
-            BlobFileInner::Native(file) => file.is_closed().await,
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(file) => file.is_closed(),
-        }
-    }
-
-    /// Read a range without moving the cursor.
-    pub async fn read_range(&self, range: Range<u64>) -> lance_core::Result<bytes::Bytes> {
-        match &self.inner {
-            BlobFileInner::Native(file) => file.read_range(range).await,
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(file) => file.read_range(range).await,
-        }
-    }
-
-    /// Read ranges without moving the cursor.
-    pub async fn read_ranges(
-        &self,
-        ranges: &[Range<u64>],
-    ) -> lance_core::Result<Vec<bytes::Bytes>> {
-        match &self.inner {
-            BlobFileInner::Native(file) => file.read_ranges(ranges).await,
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(file) => file.read_ranges(ranges).await,
-        }
-    }
-
-    /// Read from the cursor to the end.
-    pub async fn read(&self) -> lance_core::Result<bytes::Bytes> {
-        match &self.inner {
-            BlobFileInner::Native(file) => file.read().await,
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(file) => file.read().await,
-        }
-    }
-
-    /// Read up to `len` bytes and advance the cursor.
-    pub async fn read_up_to(&self, len: usize) -> lance_core::Result<bytes::Bytes> {
-        match &self.inner {
-            BlobFileInner::Native(file) => file.read_up_to(len).await,
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(file) => file.read_up_to(len).await,
-        }
-    }
-
-    /// Move the cursor to `new_cursor`.
-    pub async fn seek(&self, new_cursor: u64) -> lance_core::Result<()> {
-        match &self.inner {
-            BlobFileInner::Native(file) => file.seek(new_cursor).await,
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(file) => file.seek(new_cursor).await,
-        }
-    }
-
-    /// Current cursor position.
-    pub async fn tell(&self) -> lance_core::Result<u64> {
-        match &self.inner {
-            BlobFileInner::Native(file) => file.tell().await,
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(file) => file.tell().await,
-        }
-    }
-
-    /// Blob length in bytes.
-    pub fn size(&self) -> u64 {
-        match &self.inner {
-            BlobFileInner::Native(file) => file.size(),
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(file) => file.size(),
-        }
-    }
-
-    /// Physical byte offset in the data file. `None` on remote handles. The
-    /// Cloud byte-range route does not expose storage layout.
-    pub fn position(&self) -> Option<u64> {
-        match &self.inner {
-            BlobFileInner::Native(file) => Some(file.position()),
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(_) => None,
-        }
-    }
-
-    /// Path of the data file holding the blob. `None` on remote handles. The
-    /// Cloud byte-range route does not expose storage layout.
-    pub fn data_path(&self) -> Option<&Path> {
-        match &self.inner {
-            BlobFileInner::Native(file) => Some(file.data_path()),
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(_) => None,
-        }
-    }
-
-    /// Native storage layout. `None` on remote handles. The Cloud byte-range
-    /// route does not expose layout.
-    pub fn kind(&self) -> Option<lance_core::datatypes::BlobKind> {
-        match &self.inner {
-            BlobFileInner::Native(file) => Some(file.kind()),
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(_) => None,
-        }
-    }
-
-    /// External URI for native handles. Remote handles do not expose storage URIs.
-    pub fn uri(&self) -> Option<&str> {
-        match &self.inner {
-            BlobFileInner::Native(file) => file.uri(),
-            #[cfg(feature = "remote")]
-            BlobFileInner::Remote(_) => None,
-        }
-    }
-}
+pub use lance::dataset::BlobFile;

 /// One row-specific blob range read request.
 ///
@@ -457,10 +264,7 @@ pub(crate) async fn take_blob_files_aligned(

    let handles = dataset.take_blobs(row_ids, column).await?;
    ensure_all_row_ids_resolved(column, row_ids.len(), handles.len())?;
-    Ok(handles
-        .into_iter()
-        .map(|handle| handle.map(Into::into))
-        .collect())
+    Ok(handles)
 }

 #[cfg(test)]
@@ -34,7 +34,7 @@ use crate::remote::{
    db::{OPT_REMOTE_API_KEY, OPT_REMOTE_HOST_OVERRIDE, OPT_REMOTE_REGION},
 };
 use lance::io::ObjectStoreParams;
-pub use lance_file::version::LanceFileVersion;
+pub use lance_encoding::version::LanceFileVersion;
 #[cfg(feature = "remote")]
 use lance_io::object_store::StorageOptions;
 use lance_io::object_store::{StorageOptionsAccessor, StorageOptionsProvider};
@@ -202,17 +202,6 @@ mod tests {
        assert_eq!(table.count_rows(None).await.unwrap(), 0);
    }

-    #[tokio::test]
-    async fn create_table_in_named_memory_database() {
-        let db = connect("memory://foo").execute().await.unwrap();
-        let batch = record_batch!(("id", Int64, [1, 2, 3])).unwrap();
-
-        let table = db.create_table("my_table", batch).execute().await.unwrap();
-
-        assert_eq!(table.uri().await.unwrap(), "memory://foo/my_table.lance");
-        assert_eq!(table.count_rows(None).await.unwrap(), 3);
-    }
-
    async fn test_create_table_with_data<T>(data: T)
    where
        T: Scannable + 'static,
@@ -12,7 +12,7 @@ use lance::dataset::refs::Ref;
 use lance::dataset::{ReadParams, WriteMode, builder::DatasetBuilder};
 use lance::io::{ObjectStore, ObjectStoreParams, WrappingObjectStore};
 use lance_datafusion::utils::StreamingWriteSource;
-use lance_file::version::LanceFileVersion;
+use lance_encoding::version::LanceFileVersion;
 use lance_io::object_store::{StorageOptionsAccessor, StorageOptionsProvider};
 use lance_table::io::commit::commit_handler_from_url;
 use object_store::local::LocalFileSystem;
@@ -1294,11 +1294,9 @@ mod tests {
    use crate::connection::ConnectRequest;
    use crate::data::scannable::Scannable;
    use crate::database::{CreateTableMode, CreateTableRequest};
-    use crate::query::QueryRequest;
-    use crate::table::{AnyQuery, WriteOptions};
+    use crate::table::WriteOptions;
    use arrow_array::{Int32Array, RecordBatch, StringArray};
    use arrow_schema::{DataType, Field, Schema};
-    use futures::TryStreamExt;
    use std::path::PathBuf;
    use tempfile::tempdir;

@@ -1378,156 +1376,6 @@ mod tests {
        assert!(!tempdir.path().join("__manifest").exists());
    }

-    /// Regression test for https://github.com/lancedb/lancedb/issues/1600.
-    ///
-    /// Opening a table used to create a separate object-store client instead of
-    /// reusing the one that successfully connected to the database.  Repeating
-    /// credential discovery made S3 table opens intermittent, especially in AWS
-    /// Lambda, and the failed open was reported as `TableNotFound`.
-    #[tokio::test]
-    async fn test_open_table_reuses_connection_object_store() {
-        let tempdir = tempdir().unwrap();
-        let uri = tempdir.path().to_str().unwrap();
-        let registry = Arc::new(lance_io::object_store::ObjectStoreRegistry::default());
-        let session = Arc::new(lance::session::Session::new(16, 16, registry.clone()));
-
-        let request = ConnectRequest {
-            uri: uri.to_string(),
-            #[cfg(feature = "remote")]
-            client_config: Default::default(),
-            options: Default::default(),
-            namespace_client_properties: Default::default(),
-            manifest_enabled: false,
-            read_consistency_interval: None,
-            session: Some(session),
-        };
-        let db = ListingDatabase::connect_with_options(&request)
-            .await
-            .unwrap();
-
-        let schema = Arc::new(Schema::new(vec![Field::new("id", DataType::Int32, false)]));
-        db.create_table(CreateTableRequest {
-            name: "test".to_string(),
-            namespace_path: vec![],
-            data: Box::new(RecordBatch::new_empty(schema)) as Box<dyn Scannable>,
-            mode: CreateTableMode::Create,
-            write_options: Default::default(),
-            location: None,
-            namespace_client: None,
-        })
-        .await
-        .unwrap();
-
-        let before_open = registry.stats();
-        for _ in 0..3 {
-            let table = db
-                .open_table(OpenTableRequest {
-                    name: "test".to_string(),
-                    namespace_path: vec![],
-                    index_cache_size: None,
-                    lance_read_params: None,
-                    location: None,
-                    namespace_client: None,
-                    managed_versioning: None,
-                })
-                .await
-                .unwrap();
-            assert_eq!(table.count_rows(None).await.unwrap(), 0);
-        }
-
-        let after_open = registry.stats();
-        assert_eq!(after_open.misses, before_open.misses);
-        assert!(after_open.hits >= before_open.hits + 3);
-    }
-
-    /// Regression test for https://github.com/lancedb/lancedb/issues/3197.
-    #[cfg(unix)]
-    #[tokio::test]
-    async fn test_open_table_follows_hugging_face_symlinks() {
-        let (tempdir, db) = setup_database().await;
-        let schema = Arc::new(Schema::new(vec![Field::new("id", DataType::Int32, false)]));
-        db.create_table(CreateTableRequest {
-            name: "test".to_string(),
-            namespace_path: vec![],
-            data: Box::new(
-                RecordBatch::try_new(schema, vec![Arc::new(Int32Array::from(vec![1, 2, 3]))])
-                    .unwrap(),
-            ) as Box<dyn Scannable>,
-            mode: CreateTableMode::Create,
-            write_options: Default::default(),
-            location: None,
-            namespace_client: None,
-        })
-        .await
-        .unwrap();
-
-        let table_dir = tempdir.path().join("test.lance");
-        let versions_dir = table_dir.join("_versions");
-        let manifest_path = std::fs::read_dir(&versions_dir)
-            .unwrap()
-            .map(|entry| entry.unwrap().path())
-            .find(|path| path.extension().is_some_and(|ext| ext == "manifest"))
-            .unwrap();
-        let data_path = std::fs::read_dir(table_dir.join("data"))
-            .unwrap()
-            .map(|entry| entry.unwrap().path())
-            .find(|path| path.extension().is_some_and(|ext| ext == "lance"))
-            .unwrap();
-
-        // Hugging Face snapshots keep dataset objects in a separate blob directory and
-        // expose them through relative symlinks.
-        let blobs_dir = tempdir.path().join("blobs");
-        std::fs::create_dir(&blobs_dir).unwrap();
-        let manifest_blob = "9b603c63d0e692e05d58be25605f2f2064cc781e5ff94fe983a405059547b816";
-        let data_blob = "be64f20e5723bd0a27cfdbdb41cf7d6fad94cd572a71973b717fb8340f4310c5";
-        std::fs::rename(&manifest_path, blobs_dir.join(manifest_blob)).unwrap();
-        std::fs::rename(&data_path, blobs_dir.join(data_blob)).unwrap();
-        std::os::unix::fs::symlink(Path::new("../../blobs").join(manifest_blob), &manifest_path)
-            .unwrap();
-        std::os::unix::fs::symlink(Path::new("../../blobs").join(data_blob), &data_path).unwrap();
-        let symlink_len = std::fs::symlink_metadata(&manifest_path).unwrap().len();
-        let target_len = std::fs::metadata(&manifest_path).unwrap().len();
-        assert_ne!(symlink_len, target_len);
-
-        drop(db);
-        let db = ListingDatabase::connect_with_options(&ConnectRequest {
-            uri: tempdir.path().to_str().unwrap().to_string(),
-            #[cfg(feature = "remote")]
-            client_config: Default::default(),
-            options: Default::default(),
-            namespace_client_properties: Default::default(),
-            manifest_enabled: false,
-            read_consistency_interval: None,
-            session: None,
-        })
-        .await
-        .unwrap();
-
-        let table = db
-            .open_table(OpenTableRequest {
-                name: "test".to_string(),
-                namespace_path: vec![],
-                index_cache_size: None,
-                lance_read_params: None,
-                location: None,
-                namespace_client: None,
-                managed_versioning: None,
-            })
-            .await
-            .unwrap();
-        let batches = table
-            .query(
-                &AnyQuery::Query(QueryRequest::default()),
-                Default::default(),
-            )
-            .await
-            .unwrap()
-            .try_collect::<Vec<_>>()
-            .await
-            .unwrap();
-        assert_eq!(batches.iter().map(RecordBatch::num_rows).sum::<usize>(), 3);
-    }
-
    #[tokio::test]
    async fn test_clone_table_basic() {
        let (_tempdir, db) = setup_database().await;
@@ -2432,7 +2280,7 @@ mod tests {

    #[tokio::test]
    async fn test_table_uri() {
-        let (_tempdir, mut db) = setup_database().await;
+        let (_tempdir, db) = setup_database().await;

        let mut pb = PathBuf::new();
        pb.push(db.uri.clone());
@@ -2441,18 +2289,6 @@ mod tests {
        let expected = pb.to_str().unwrap();
        let uri = db.table_uri("test").ok().unwrap();
        assert_eq!(uri, expected);
-
-        // URI paths always use forward slashes, even on Windows. Using
-        // `Path::join` here used to produce `az://container/prefix\\test.lance`,
-        // which Azure treated as a different object from the table returned by
-        // `table_names` (https://github.com/lancedb/lancedb/issues/1072).
-        for base_uri in ["az://container/prefix", "az://container/prefix/"] {
-            db.uri = base_uri.to_string();
-            assert_eq!(
-                db.table_uri("test").unwrap(),
-                "az://container/prefix/test.lance"
-            );
-        }
    }

    /// Regression: connecting via a URL-style URI (which goes through
@@ -201,7 +201,7 @@ impl LanceNamespaceDatabase {
        &self,
        request: &DbCreateTableRequest,
    ) -> Result<(
-        Option<lance_file::version::LanceFileVersion>,
+        Option<lance_encoding::version::LanceFileVersion>,
        Option<bool>,
        Option<bool>,
    )> {
@@ -214,7 +214,7 @@ impl LanceNamespaceDatabase {

        let storage_version_override = storage_options
            .and_then(|opts| opts.get(OPT_NEW_TABLE_STORAGE_VERSION))
-            .map(|s| s.parse::<lance_file::version::LanceFileVersion>())
+            .map(|s| s.parse::<lance_encoding::version::LanceFileVersion>())
            .transpose()?;

        let v2_manifest_override = storage_options
@@ -169,12 +169,6 @@ impl From<DataFusionError> for Error {

 impl From<lance::Error> for Error {
    fn from(source: lance::Error) -> Self {
-        if has_unsupported_local_filesystem_source(&source) {
-            return Self::NotSupported {
-                message: "the filesystem does not support an operation required for safe Lance commits (such as atomic rename). Object-storage mounts such as Mountpoint for Amazon S3 are not supported; use the native object-store URI (for example, s3://bucket/path) instead".to_string(),
-            };
-        }
-
        // Try to unwrap external errors that were wrapped by lance
        match source {
            lance::Error::Wrapped { error, .. } => Self::from_box_error(error),
@@ -187,27 +181,6 @@ impl From<lance::Error> for Error {
    }
 }

-fn has_unsupported_local_filesystem_source(error: &(dyn std::error::Error + 'static)) -> bool {
-    let mut current = Some(error);
-    let mut is_local_filesystem = false;
-    let mut is_unsupported = false;
-    while let Some(error) = current {
-        is_local_filesystem |= error
-            .downcast_ref::<object_store::Error>()
-            .is_some_and(|error| {
-                matches!(error, object_store::Error::Generic { store, .. } if *store == "LocalFileSystem")
-            });
-        is_unsupported |= error
-            .downcast_ref::<std::io::Error>()
-            .is_some_and(|error| error.kind() == std::io::ErrorKind::Unsupported);
-        if is_local_filesystem && is_unsupported {
-            return true;
-        }
-        current = error.source();
-    }
-    false
-}
-
 impl Error {
    fn from_box_error(mut source: Box<dyn std::error::Error + Send + Sync>) -> Self {
        source = match source.downcast::<Self>() {
@@ -297,46 +270,3 @@ impl From<candle_core::Error> for Error {
        }
    }
 }
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[test]
-    fn unsupported_filesystem_operations_have_actionable_error() {
-        let object_store_error = object_store::Error::Generic {
-            store: "LocalFileSystem",
-            source: Box::new(std::io::Error::from(std::io::ErrorKind::Unsupported)),
-        };
-        let lance_error = lance::Error::io_source(Box::new(object_store_error));
-
-        let error = Error::from(lance_error);
-
-        assert!(matches!(
-            error,
-            Error::NotSupported { message }
-                if message.contains("Mountpoint for Amazon S3")
-                    && message.contains("s3://bucket/path")
-        ));
-    }
-
-    #[test]
-    fn other_io_errors_remain_lance_errors() {
-        let object_store_error = object_store::Error::Generic {
-            store: "LocalFileSystem",
-            source: Box::new(std::io::Error::from(std::io::ErrorKind::PermissionDenied)),
-        };
-        let lance_error = lance::Error::io_source(Box::new(object_store_error));
-
-        assert!(matches!(Error::from(lance_error), Error::Lance { .. }));
-    }
-
-    #[test]
-    fn unsupported_non_filesystem_errors_remain_lance_errors() {
-        let lance_error = lance::Error::io_source(Box::new(std::io::Error::from(
-            std::io::ErrorKind::Unsupported,
-        )));
-
-        assert!(matches!(Error::from(lance_error), Error::Lance { .. }));
-    }
-}
@@ -132,14 +132,9 @@ impl ObjectStore for MirroringObjectStore {
        if to.primary_only() {
            self.primary.copy_opts(from, to, options).await
        } else {
-            // The secondary store can be process-local and less durable than the
-            // primary, so a source written by another process may not exist here
-            // or may be evicted before the copy begins.
-            match self.secondary.copy_opts(from, to, options.clone()).await {
-                Ok(()) | Err(Error::NotFound { .. }) => {}
-                Err(err) => return Err(err),
-            }
-            self.primary.copy_opts(from, to, options).await
+            self.secondary.copy_opts(from, to, options.clone()).await?;
+            self.primary.copy_opts(from, to, options).await?;
+            Ok(())
        }
    }
 }
@@ -197,8 +192,7 @@ mod test {
    use futures::TryStreamExt;
    use lance::{dataset::WriteParams, io::ObjectStoreParams};
    use lance_testing::datagen::{BatchGenerator, IncrementingInt32, RandomVector};
-    use object_store::{local::LocalFileSystem, memory::InMemory};
-    use std::time::Duration;
+    use object_store::local::LocalFileSystem;
    use tempfile;

    use crate::{
@@ -207,139 +201,6 @@ mod test {
        table::WriteOptions,
    };

-    #[derive(Debug)]
-    struct EvictBeforeCopyStore {
-        inner: Arc<dyn ObjectStore>,
-    }
-
-    impl std::fmt::Display for EvictBeforeCopyStore {
-        fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
-            write!(f, "EvictBeforeCopyStore")
-        }
-    }
-
-    #[async_trait]
-    impl ObjectStore for EvictBeforeCopyStore {
-        async fn put_opts(
-            &self,
-            location: &Path,
-            payload: PutPayload,
-            options: PutOptions,
-        ) -> Result<PutResult> {
-            self.inner.put_opts(location, payload, options).await
-        }
-
-        async fn put_multipart_opts(
-            &self,
-            location: &Path,
-            options: PutMultipartOptions,
-        ) -> Result<Box<dyn MultipartUpload>> {
-            self.inner.put_multipart_opts(location, options).await
-        }
-
-        async fn get_opts(&self, location: &Path, options: GetOptions) -> Result<GetResult> {
-            self.inner.get_opts(location, options).await
-        }
-
-        fn delete_stream(
-            &self,
-            locations: BoxStream<'static, Result<Path>>,
-        ) -> BoxStream<'static, Result<Path>> {
-            self.inner.delete_stream(locations)
-        }
-
-        fn list(&self, prefix: Option<&Path>) -> BoxStream<'static, Result<ObjectMeta>> {
-            self.inner.list(prefix)
-        }
-
-        async fn list_with_delimiter(&self, prefix: Option<&Path>) -> Result<ListResult> {
-            self.inner.list_with_delimiter(prefix).await
-        }
-
-        async fn copy_opts(&self, from: &Path, to: &Path, options: CopyOptions) -> Result<()> {
-            self.inner.delete(from).await?;
-            self.inner.copy_opts(from, to, options).await
-        }
-    }
-
-    #[tokio::test]
-    async fn test_copy_when_source_is_missing_from_secondary() {
-        let primary_dir = tempfile::tempdir().unwrap();
-        let secondary_dir = tempfile::tempdir().unwrap();
-        let primary: Arc<dyn ObjectStore> =
-            Arc::new(LocalFileSystem::new_with_prefix(primary_dir.path()).unwrap());
-        let secondary: Arc<dyn ObjectStore> =
-            Arc::new(LocalFileSystem::new_with_prefix(secondary_dir.path()).unwrap());
-        let store = MirroringObjectStore {
-            primary: primary.clone(),
-            secondary: secondary.clone(),
-        };
-        let staging = Path::from("_versions/1.manifest-staging");
-        let finalized = Path::from("_versions/1.manifest");
-
-        primary
-            .put(&staging, "manifest contents".into())
-            .await
-            .unwrap();
-
-        tokio::time::timeout(Duration::from_secs(5), store.copy(&staging, &finalized))
-            .await
-            .expect("copy should not hang when the secondary source is missing")
-            .unwrap();
-
-        let copied = primary
-            .get(&finalized)
-            .await
-            .unwrap()
-            .bytes()
-            .await
-            .unwrap();
-        assert_eq!(copied, "manifest contents");
-        assert!(matches!(
-            secondary.head(&finalized).await,
-            Err(Error::NotFound { .. })
-        ));
-    }
-
-    #[tokio::test]
-    async fn test_copy_when_secondary_source_disappears_after_head() {
-        let primary: Arc<dyn ObjectStore> = Arc::new(InMemory::new());
-        let secondary_inner: Arc<dyn ObjectStore> = Arc::new(InMemory::new());
-        let secondary: Arc<dyn ObjectStore> = Arc::new(EvictBeforeCopyStore {
-            inner: secondary_inner.clone(),
-        });
-        let store = MirroringObjectStore {
-            primary: primary.clone(),
-            secondary,
-        };
-        let staging = Path::from("_versions/1.manifest-staging");
-        let finalized = Path::from("_versions/1.manifest");
-
-        primary
-            .put(&staging, "manifest contents".into())
-            .await
-            .unwrap();
-        secondary_inner
-            .put(&staging, "manifest contents".into())
-            .await
-            .unwrap();
-
-        store.copy(&staging, &finalized).await.unwrap();
-
-        let copied = primary
-            .get(&finalized)
-            .await
-            .unwrap()
-            .bytes()
-            .await
-            .unwrap();
-        assert_eq!(copied, "manifest contents");
-        assert!(matches!(
-            secondary_inner.head(&finalized).await,
-            Err(Error::NotFound { .. })
-        ));
-    }
-
    // This test is ignored because lance 3.0 introduced LocalWriter optimization
    // that bypasses the object store wrapper for local writes. The mirroring feature
    // still works for remote/cloud storage, but can't be tested with local storage.
@@ -1661,8 +1661,14 @@ mod tests {

    #[tokio::test]
    async fn test_setters_getters() {
+        // TODO: Switch back to memory://foo after https://github.com/lancedb/lancedb/issues/1051
+        // is fixed
+        let tmp_dir = tempdir().unwrap();
+        let dataset_path = tmp_dir.path().join("test.lance");
+        let uri = dataset_path.to_str().unwrap();
+
        let batches = make_test_batches();
-        let conn = connect("memory://foo").execute().await.unwrap();
+        let conn = connect(uri).execute().await.unwrap();
        let table = conn
            .create_table("my_table", batches)
            .execute()
@@ -1757,8 +1763,14 @@ mod tests {

    #[tokio::test]
    async fn test_execute() {
+        // TODO: Switch back to memory://foo after https://github.com/lancedb/lancedb/issues/1051
+        // is fixed
+        let tmp_dir = tempdir().unwrap();
+        let dataset_path = tmp_dir.path().join("test.lance");
+        let uri = dataset_path.to_str().unwrap();
+
        let batches = make_non_empty_batches();
-        let conn = connect("memory://foo").execute().await.unwrap();
+        let conn = connect(uri).execute().await.unwrap();
        let table = conn
            .create_table("my_table", batches)
            .execute()
@@ -1877,8 +1889,14 @@ mod tests {

    #[tokio::test]
    async fn test_select_with_transform() {
+        // TODO: Switch back to memory://foo after https://github.com/lancedb/lancedb/issues/1051
+        // is fixed
+        let tmp_dir = tempdir().unwrap();
+        let dataset_path = tmp_dir.path().join("test.lance");
+        let uri = dataset_path.to_str().unwrap();
+
        let batches = make_non_empty_batches();
-        let conn = connect("memory://foo").execute().await.unwrap();
+        let conn = connect(uri).execute().await.unwrap();
        let table = conn
            .create_table("my_table", batches)
            .execute()
@@ -1975,9 +1993,15 @@ mod tests {

    #[tokio::test]
    async fn test_execute_no_vector() {
+        // TODO: Switch back to memory://foo after https://github.com/lancedb/lancedb/issues/1051
+        // is fixed
+        let tmp_dir = tempdir().unwrap();
+        let dataset_path = tmp_dir.path().join("test.lance");
+        let uri = dataset_path.to_str().unwrap();
+
        // test that it's ok to not specify a query vector (just filter / limit)
        let batches = make_non_empty_batches();
-        let conn = connect("memory://foo").execute().await.unwrap();
+        let conn = connect(uri).execute().await.unwrap();
        let table = conn
            .create_table("my_table", batches)
            .execute()
@@ -373,37 +373,6 @@ pub fn parse_db_url(db_url: &str) -> Result<ParsedDbUrl> {
    Ok(ParsedDbUrl { db_name, db_prefix })
 }

-fn validate_dns_hostname(hostname: &str) -> Result<()> {
-    let ascii_hostname = match url::Host::parse(hostname) {
-        Ok(url::Host::Domain(hostname)) => hostname,
-        Ok(_) => {
-            return Err(Error::InvalidInput {
-                message: "LanceDB Cloud database URI or region produced a non-DNS hostname"
-                    .to_string(),
-            });
-        }
-        Err(err) => {
-            return Err(Error::InvalidInput {
-                message: format!(
-                    "LanceDB Cloud database URI or region produced an invalid hostname: {err}"
-                ),
-            });
-        }
-    };
-
-    if ascii_hostname.len() > 253
-        || ascii_hostname
-            .split('.')
-            .any(|label| label.is_empty() || label.len() > 63)
-    {
-        return Err(Error::InvalidInput {
-            message: "LanceDB Cloud database URI or region produced an invalid hostname: DNS labels must contain 1 to 63 bytes and the full hostname must not exceed 253 bytes".to_string(),
-        });
-    }
-
-    Ok(())
-}
-
 impl RestfulLanceDbClient<Sender> {
    fn get_timeout(passed: Option<Duration>, env_var: &str) -> Result<Option<Duration>> {
        if let Some(passed) = passed {
@@ -511,11 +480,7 @@ impl RestfulLanceDbClient<Sender> {

        let host = match host_override {
            Some(host_override) => host_override,
-            None => {
-                let hostname = format!("{}.{}.api.lancedb.com", parsed_url.db_name, region);
-                validate_dns_hostname(&hostname)?;
-                format!("https://{hostname}")
-            }
+            None => format!("https://{}.{}.api.lancedb.com", parsed_url.db_name, region),
        };
        debug!("Created client for host: {}", host);
        let retry_config = client_config.retry_config.clone().try_into()?;
@@ -1192,29 +1157,6 @@ mod tests {
        assert_eq!(headers.get("x-api-key").unwrap(), "api-key");
    }

-    #[test]
-    fn test_rejects_invalid_cloud_dns_hostname() {
-        let invalid_database_names = ["a".repeat(64), "invalid..database".to_string()];
-
-        for db_name in invalid_database_names {
-            let parsed_url = parse_db_url(&format!("db://{db_name}")).unwrap();
-            let error = RestfulLanceDbClient::<Sender>::try_new(
-                &parsed_url,
-                "us-east-1",
-                None,
-                HeaderMap::new(),
-                ClientConfig::default(),
-                None,
-            )
-            .unwrap_err();
-
-            assert!(
-                matches!(error, Error::InvalidInput { ref message } if message.contains("DNS labels must contain 1 to 63 bytes")),
-                "unexpected error: {error}"
-            );
-        }
-    }
-
    // Test implementation of HeaderProvider
    #[derive(Debug, Clone)]
    struct TestHeaderProvider {
@@ -1,7 +1,7 @@
 // SPDX-License-Identifier: Apache-2.0
 // SPDX-FileCopyrightText: Copyright The LanceDB Authors

-pub mod blobs;
+mod blobs;
 pub mod insert;

 use self::insert::{RemoteWriteExec, WriteOp};
@@ -2791,10 +2791,9 @@ impl<S: HttpSend> BaseTable for RemoteTable<S> {
    }

    async fn index_stats(&self, index_name: &str) -> Result<Option<IndexStatistics>> {
-        let encoded_name = urlencoding::encode(index_name);
        let mut request = self.post_read(&format!(
-            "/v1/table/{}/index/{encoded_name}/stats/",
-            self.identifier
+            "/v1/table/{}/index/{}/stats/",
+            self.identifier, index_name
        ));
        let version = self.current_version().await;
        let mut body = serde_json::json!({ "version": version });
@@ -2821,10 +2820,9 @@ impl<S: HttpSend> BaseTable for RemoteTable<S> {
    }

    async fn drop_index(&self, index_name: &str) -> Result<()> {
-        let encoded_name = urlencoding::encode(index_name);
        let request = self.apply_branch_query(self.client.post(&format!(
-            "/v1/table/{}/index/{encoded_name}/drop/",
-            self.identifier
+            "/v1/table/{}/index/{}/drop/",
+            self.identifier, index_name
        )));
        let (request_id, response) = self.send(request, true).await?;
        if response.status() == StatusCode::NOT_FOUND {
@@ -2837,10 +2835,9 @@ impl<S: HttpSend> BaseTable for RemoteTable<S> {
    }

    async fn prewarm_index(&self, index_name: &str) -> Result<()> {
-        let encoded_name = urlencoding::encode(index_name);
        let request = self.client.post(&format!(
-            "/v1/table/{}/index/{encoded_name}/prewarm/",
-            self.identifier
+            "/v1/table/{}/index/{}/prewarm/",
+            self.identifier, index_name
        ));
        let (request_id, response) = self.send(request, true).await?;
        if response.status() == StatusCode::NOT_FOUND {
@@ -3092,12 +3089,10 @@ mod tests {
            Box::pin(table.delete("false").map_ok(|_| ())),
            Box::pin(
                table
-                    .add_columns()
-                    .transform(NewColumnTransform::SqlExpressions(vec![(
-                        "x".into(),
-                        "y".into(),
-                    )]))
-                    .execute()
+                    .add_columns(
+                        NewColumnTransform::SqlExpressions(vec![("x".into(), "y".into())]),
+                        None,
+                    )
                    .map_ok(|_| ()),
            ),
            Box::pin(async {
@@ -4305,9 +4300,32 @@ mod tests {
            "fetch_blobs",
        );

+        let message = table
+            .fetch_blob_files("image", &[1])
+            .await
+            .unwrap_err()
+            .to_string();
+        assert!(
+            message.contains("fetch_blob_files is not supported on LanceDB Cloud"),
+            "got: {message}"
+        );
+        assert!(
+            !message.contains("Use fetch_blobs"),
+            "old server must not be told to use fetch_blobs, got: {message}"
+        );
+    }
+
+    #[tokio::test]
+    async fn test_blob_files_point_at_fetch_blobs_on_a_blob_capable_server() {
+        let table = Table::new_with_handler_version(
+            "my_table",
+            semver::Version::new(0, 5, 0),
+            |_| -> http::Response<String> { panic!("fetch_blob_files must not reach the server") },
+        );
+
        assert_not_supported_error(
            table.fetch_blob_files("image", &[1]).await.unwrap_err(),
-            "requires LanceDB Cloud server 0.5.0 or newer",
+            "Use fetch_blobs for full bytes",
        );
    }

@@ -5907,18 +5925,16 @@ mod tests {
            .await
            .unwrap();

-        // Positions are relative to the first retained token, so dropping the
-        // leading "hello" stop word does not shift the remaining tokens.
        assert_eq!(
            tokens,
            vec![
                FtsToken {
                    text: "こんにちは".to_string(),
-                    position: 0,
+                    position: 1,
                },
                FtsToken {
                    text: "世界".to_string(),
-                    position: 1,
+                    position: 2,
                },
            ]
        );
@@ -6395,12 +6411,13 @@ mod tests {
        });

        let result = table
-            .add_columns()
-            .transform(NewColumnTransform::SqlExpressions(vec![
-                ("b".into(), "a + 1".into()),
-                ("x".into(), "cast(NULL as int32)".into()),
-            ]))
-            .execute()
+            .add_columns(
+                NewColumnTransform::SqlExpressions(vec![
+                    ("b".into(), "a + 1".into()),
+                    ("x".into(), "cast(NULL as int32)".into()),
+                ]),
+                None,
+            )
            .await
            .unwrap();

@@ -6494,41 +6511,6 @@ mod tests {
        assert!(matches!(e, Error::IndexNotFound { .. }));
    }

-    /// Index names are unvalidated, so reserved characters must be
-    /// percent-encoded or they restructure the request path.
-    #[tokio::test]
-    async fn test_per_index_paths_encode_reserved_characters() {
-        const NAME: &str = "my/index?a#b c";
-        const PREFIX: &str = "/v1/table/my_table/index/my%2Findex%3Fa%23b%20c";
-
-        let table = Table::new_with_handler("my_table", |request| {
-            assert_eq!(request.url().path(), format!("{PREFIX}/stats/"));
-            let body = serde_json::json!({
-              "num_indexed_rows": 1,
-              "num_unindexed_rows": 0,
-              "index_type": "IVF_PQ",
-              "distance_type": "l2"
-            });
-            http::Response::builder()
-                .status(200)
-                .body(serde_json::to_string(&body).unwrap())
-                .unwrap()
-        });
-        assert!(table.index_stats(NAME).await.unwrap().is_some());
-
-        let table = Table::new_with_handler("my_table", |request| {
-            assert_eq!(request.url().path(), format!("{PREFIX}/drop/"));
-            http::Response::builder().status(200).body("{}").unwrap()
-        });
-        table.drop_index(NAME).await.unwrap();
-
-        let table = Table::new_with_handler("my_table", |request| {
-            assert_eq!(request.url().path(), format!("{PREFIX}/prewarm/"));
-            http::Response::builder().status(200).body("{}").unwrap()
-        });
-        table.prewarm_index(NAME).await.unwrap();
-    }
-
    #[tokio::test]
    async fn test_set_lsm_write_spec_unsharded() {
        let table = Table::new_with_handler("my_table", |request| {
@@ -7160,12 +7142,10 @@ mod tests {
            }
            "add_columns" => {
                let _ = table
-                    .add_columns()
-                    .transform(NewColumnTransform::SqlExpressions(vec![(
-                        "c".into(),
-                        "a + 1".into(),
-                    )]))
-                    .execute()
+                    .add_columns(
+                        NewColumnTransform::SqlExpressions(vec![("c".into(), "a + 1".into())]),
+                        None,
+                    )
                    .await;
            }
            "drop_columns" => {
@@ -9923,12 +9903,10 @@ mod tests {
            .await
            .unwrap();
        branch
-            .add_columns()
-            .transform(NewColumnTransform::SqlExpressions(vec![(
-                "b".into(),
-                "a + 1".into(),
-            )]))
-            .execute()
+            .add_columns(
+                NewColumnTransform::SqlExpressions(vec![("b".into(), "a + 1".into())]),
+                None,
+            )
            .await
            .unwrap();
        branch
@@ -3,7 +3,6 @@

 //! LanceDB Table APIs

-use crate::blob::BlobFile;
 use arrow_array::{LargeBinaryArray, RecordBatch, RecordBatchReader};
 use arrow_schema::{Schema, SchemaRef};
 use async_trait::async_trait;
@@ -13,6 +12,7 @@ use datafusion_physical_plan::ExecutionPlan;
 use datafusion_physical_plan::display::DisplayableExecutionPlan;
 use futures::StreamExt;
 use futures::stream::FuturesUnordered;
+use lance::dataset::BlobFile;
 pub use lance::dataset::ColumnAlteration;
 pub use lance::dataset::NewColumnTransform;
 pub use lance::dataset::ReadParams;
@@ -65,7 +65,6 @@ use crate::utils::{PatchReadParam, PatchWriteParam, resolve_arrow_field_path};
 use self::dataset::DatasetConsistencyWrapper;
 use self::merge::MergeInsertBuilder;

-pub mod add_columns;
 mod add_data;
 pub mod branch_merge;
 mod create_index;
@@ -80,7 +79,6 @@ pub mod schema_evolution;
 pub mod update;
 pub mod write_progress;
 use crate::index::waiter::wait_for_index;
-pub use add_columns::AddColumnsBuilder;
 #[cfg(feature = "remote")]
 pub(crate) use add_data::PreprocessingOutput;
 pub use add_data::{AddDataBuilder, AddDataMode, AddResult, NaNVectorBehavior};
@@ -1622,8 +1620,12 @@ impl Table {
    }

    /// Add new columns to the table, providing values to fill in.
-    pub fn add_columns(&self) -> AddColumnsBuilder {
-        AddColumnsBuilder::new(self.inner.clone())
+    pub async fn add_columns(
+        &self,
+        transforms: NewColumnTransform,
+        read_columns: Option<Vec<String>>,
+    ) -> Result<AddColumnsResult> {
+        self.inner.add_columns(transforms, read_columns).await
    }

    /// Change a column's name or nullability.
@@ -1,161 +0,0 @@
-// SPDX-License-Identifier: Apache-2.0
-// SPDX-FileCopyrightText: Copyright The LanceDB Authors
-
-//! Builder for adding columns to a table.
-
-use std::sync::Arc;
-
-use lance::dataset::NewColumnTransform;
-
-use super::BaseTable;
-use super::schema_evolution::AddColumnsResult;
-use crate::{Error, Result};
-
-/// Adds columns to a table. See [`Table::add_columns`](super::Table::add_columns).
-pub struct AddColumnsBuilder {
-    parent: Arc<dyn BaseTable>,
-    transform: Option<NewColumnTransform>,
-    read_columns: Option<Vec<String>>,
-}
-
-impl std::fmt::Debug for AddColumnsBuilder {
-    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
-        f.debug_struct("AddColumnsBuilder")
-            .field("parent", &self.parent)
-            .field("has_transform", &self.transform.is_some())
-            .field("read_columns", &self.read_columns)
-            .finish()
-    }
-}
-
-impl AddColumnsBuilder {
-    pub(crate) fn new(parent: Arc<dyn BaseTable>) -> Self {
-        Self {
-            parent,
-            transform: None,
-            read_columns: None,
-        }
-    }
-
-    /// Set how the new columns' values are produced. Required.
-    pub fn transform(mut self, transform: NewColumnTransform) -> Self {
-        self.transform = Some(transform);
-        self
-    }
-
-    /// Limit which existing columns a [`NewColumnTransform::BatchUDF`] mapper
-    /// receives. Every other transform determines what it reads, so setting
-    /// this alongside one is an error rather than a silent no-op.
-    pub fn read_columns(mut self, columns: impl IntoIterator<Item = impl Into<String>>) -> Self {
-        self.read_columns = Some(columns.into_iter().map(Into::into).collect());
-        self
-    }
-
-    /// Add the columns.
-    pub async fn execute(self) -> Result<AddColumnsResult> {
-        let Self {
-            parent,
-            transform,
-            read_columns,
-        } = self;
-
-        let Some(transform) = transform else {
-            return Err(Error::InvalidInput {
-                message: "add_columns requires a transform".into(),
-            });
-        };
-
-        if read_columns.is_some() && !matches!(transform, NewColumnTransform::BatchUDF(_)) {
-            return Err(Error::InvalidInput {
-                message: "read_columns applies only to a BatchUDF transform; \
-                          every other transform determines what it reads"
-                    .into(),
-            });
-        }
-
-        parent.add_columns(transform, read_columns).await
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use std::sync::Arc;
-
-    use arrow_array::{Int32Array, RecordBatch, record_batch};
-    use arrow_schema::{DataType, Field, Schema};
-    use lance::dataset::{BatchUDF, NewColumnTransform};
-
-    use crate::Table;
-    use crate::connect;
-
-    async fn table_with_two_columns(name: &str) -> Table {
-        let conn = connect("memory://").execute().await.unwrap();
-        let batch = record_batch!(("x", Int32, [1, 2, 3]), ("y", Int32, [10, 20, 30])).unwrap();
-        conn.create_table(name, batch).execute().await.unwrap()
-    }
-
-    #[tokio::test]
-    async fn test_requires_a_transform() {
-        let table = table_with_two_columns("no_transform").await;
-        let err = table.add_columns().execute().await.unwrap_err();
-        assert!(
-            err.to_string().contains("requires a transform"),
-            "got: {err}"
-        );
-    }
-
-    #[tokio::test]
-    async fn test_read_columns_with_sql_expressions_is_rejected() {
-        let table = table_with_two_columns("read_cols_sql").await;
-        let err = table
-            .add_columns()
-            .transform(NewColumnTransform::SqlExpressions(vec![(
-                "doubled".into(),
-                "x * 2".into(),
-            )]))
-            .read_columns(["x"])
-            .execute()
-            .await
-            .unwrap_err();
-        assert!(err.to_string().contains("BatchUDF"), "got: {err}");
-
-        let schema = table.schema().await.unwrap();
-        assert!(
-            schema.field_with_name("doubled").is_err(),
-            "a rejected call must not commit"
-        );
-    }
-
-    #[tokio::test]
-    async fn test_read_columns_limits_what_a_batch_udf_sees() {
-        let table = table_with_two_columns("read_cols_udf").await;
-
-        let output_schema = Arc::new(Schema::new(vec![Field::new("sum", DataType::Int32, true)]));
-        let mapper_schema = output_schema.clone();
-        let udf = BatchUDF {
-            mapper: Box::new(move |batch: &RecordBatch| {
-                assert!(batch.column_by_name("x").is_some());
-                assert!(batch.column_by_name("y").is_none(), "y was not requested");
-                let x = batch["x"].as_any().downcast_ref::<Int32Array>().unwrap();
-                let doubled: Int32Array = x.iter().map(|v| v.map(|v| v * 2)).collect();
-                Ok(RecordBatch::try_new(
-                    mapper_schema.clone(),
-                    vec![Arc::new(doubled)],
-                )?)
-            }),
-            output_schema,
-            result_checkpoint: None,
-        };
-
-        table
-            .add_columns()
-            .transform(NewColumnTransform::BatchUDF(udf))
-            .read_columns(["x"])
-            .execute()
-            .await
-            .unwrap();
-
-        let schema = table.schema().await.unwrap();
-        assert!(schema.field_with_name("sum").is_ok());
-    }
-}
@@ -576,12 +576,10 @@ mod tests {

        // Add a new physical column AFTER the embedding column.
        table
-            .add_columns()
-            .transform(NewColumnTransform::SqlExpressions(vec![(
-                "score".into(),
-                "42.0".into(),
-            )]))
-            .execute()
+            .add_columns(
+                NewColumnTransform::SqlExpressions(vec![("score".into(), "42.0".into())]),
+                None,
+            )
            .await
            .unwrap();

@@ -685,9 +683,7 @@ mod tests {
            true,
        )]));
        table
-            .add_columns()
-            .transform(NewColumnTransform::AllNulls(nested_schema))
-            .execute()
+            .add_columns(NewColumnTransform::AllNulls(nested_schema), None)
            .await
            .unwrap();

@@ -315,10 +315,7 @@ pub(crate) async fn execute_merge_insert(

 #[cfg(test)]
 mod tests {
-    use arrow_array::builder::FixedSizeBinaryBuilder;
-    use arrow_array::{
-        Int32Array, RecordBatch, RecordBatchIterator, RecordBatchReader, StringArray, UInt64Array,
-    };
+    use arrow_array::{Int32Array, RecordBatch, RecordBatchIterator, RecordBatchReader};
    use arrow_schema::{DataType, Field, Schema};
    use std::sync::Arc;

@@ -340,42 +337,6 @@ mod tests {
        Box::new(RecordBatchIterator::new(vec![Ok(batch)], schema))
    }

-    fn fixed_size_binary_merge_batch(
-        id_range: std::ops::Range<u64>,
-        price: u64,
-    ) -> Box<dyn RecordBatchReader + Send> {
-        let ids = id_range.collect::<Vec<_>>();
-        let mut id_builder = FixedSizeBinaryBuilder::new(16);
-        for id in &ids {
-            let mut bytes = [0; 16];
-            bytes[..8].copy_from_slice(&id.to_le_bytes());
-            id_builder.append_value(bytes).unwrap();
-        }
-
-        let schema = Arc::new(Schema::new(vec![
-            Field::new("id", DataType::FixedSizeBinary(16), false),
-            Field::new("id_as_int", DataType::UInt64, false),
-            Field::new("name", DataType::Utf8, false),
-            Field::new("market", DataType::Utf8, false),
-        ]));
-        let batch = RecordBatch::try_new(
-            schema.clone(),
-            vec![
-                Arc::new(id_builder.finish()),
-                Arc::new(UInt64Array::from_iter_values(ids.iter().copied())),
-                Arc::new(StringArray::from_iter_values(
-                    ids.iter().map(|id| format!("name{id}")),
-                )),
-                Arc::new(StringArray::from_iter_values(std::iter::repeat_n(
-                    format!("market_{price}"),
-                    ids.len(),
-                ))),
-            ],
-        )
-        .unwrap();
-        Box::new(RecordBatchIterator::new(vec![Ok(batch)], schema))
-    }
-
    #[tokio::test]
    async fn test_merge_insert() {
        let conn = connect("memory://").execute().await.unwrap();
@@ -427,36 +388,6 @@ mod tests {
        );
    }

-    #[tokio::test]
-    async fn test_merge_insert_fixed_size_binary_non_nullable() {
-        // Regression test for #2869: an unrelated FixedSizeBinary column used to corrupt the
-        // outer join that implements when_not_matched_by_source_delete.
-        let conn = connect("memory://").execute().await.unwrap();
-        let table = conn
-            .create_table(
-                "fixed_size_binary_merge",
-                fixed_size_binary_merge_batch(0..256, 100),
-            )
-            .execute()
-            .await
-            .unwrap();
-
-        let mut merge_insert = table.merge_insert(&["id_as_int"]);
-        merge_insert
-            .when_matched_update_all(None)
-            .when_not_matched_insert_all()
-            .when_not_matched_by_source_delete(None);
-        let result = merge_insert
-            .execute(fixed_size_binary_merge_batch(100..356, 200))
-            .await
-            .unwrap();
-
-        assert_eq!(result.num_updated_rows, 156);
-        assert_eq!(result.num_inserted_rows, 100);
-        assert_eq!(result.num_deleted_rows, 100);
-        assert_eq!(table.count_rows(None).await.unwrap(), 256);
-    }
-
    #[tokio::test]
    async fn test_merge_insert_use_index() {
        let conn = connect("memory://").execute().await.unwrap();
@@ -18,6 +18,7 @@

 use std::collections::HashMap;
 use std::sync::Arc;
+
 use std::time::Duration;

 use arrow_array::cast::AsArray;
@@ -214,17 +214,12 @@ pub(crate) async fn execute_optimize(

 #[cfg(test)]
 mod tests {
-    use arrow_array::{
-        Array, FixedSizeListArray, Float32Array, Int32Array, RecordBatch, StringArray,
-    };
+    use arrow_array::{Int32Array, RecordBatch, StringArray};
    use arrow_schema::{DataType, Field, Schema};
-    use lance_arrow::FixedSizeListArrayExt;
    use rstest::rstest;
    use std::sync::Arc;

    use crate::connect;
-    use crate::database::listing::OPT_NEW_TABLE_ENABLE_STABLE_ROW_IDS;
-    use crate::index::vector::IvfRqIndexBuilder;
    use crate::index::{Index, scalar::BTreeIndexBuilder};
    use crate::query::ExecutableQuery;
    use crate::table::{CompactionOptions, OptimizeAction, OptimizeStats};
@@ -309,96 +304,6 @@ mod tests {
        assert_eq!(all_values, expected);
    }

-    #[tokio::test]
-    async fn test_compact_with_concurrent_add() {
-        const NUM_FRAGMENTS: usize = 5;
-        const ROWS_PER_FRAGMENT: i32 = 300;
-
-        let tmpdir = tempfile::tempdir().unwrap();
-        let conn = connect(tmpdir.path().to_str().unwrap())
-            .execute()
-            .await
-            .unwrap();
-        let schema = Arc::new(Schema::new(vec![Field::new("id", DataType::Int32, false)]));
-        let batch = RecordBatch::try_new(
-            schema,
-            vec![Arc::new(Int32Array::from_iter_values(0..ROWS_PER_FRAGMENT))],
-        )
-        .unwrap();
-
-        let table = conn
-            .create_table("test_concurrent_compact", batch.clone())
-            .execute()
-            .await
-            .unwrap();
-        table
-            .create_index(&["id"], Index::BTree(BTreeIndexBuilder::default()))
-            .execute()
-            .await
-            .unwrap();
-        for _ in 0..NUM_FRAGMENTS {
-            table.add(batch.clone()).execute().await.unwrap();
-        }
-
-        // Use separate handles so the two writes actually overlap, as they can
-        // when different Node connections operate on the same S3 table.
-        let compact_table = conn
-            .open_table("test_concurrent_compact")
-            .execute()
-            .await
-            .unwrap();
-        let append_table = conn
-            .open_table("test_concurrent_compact")
-            .execute()
-            .await
-            .unwrap();
-        let compact_task = tokio::spawn(async move {
-            compact_table
-                .optimize(OptimizeAction::Compact {
-                    options: CompactionOptions {
-                        target_rows_per_fragment: 1_000,
-                        ..Default::default()
-                    },
-                    remap_options: None,
-                })
-                .await
-        });
-        tokio::task::yield_now().await;
-        for _ in 0..NUM_FRAGMENTS {
-            append_table.add(batch.clone()).execute().await.unwrap();
-        }
-        compact_task.await.unwrap().unwrap();
-
-        let table = conn
-            .open_table("test_concurrent_compact")
-            .execute()
-            .await
-            .unwrap();
-        let dataset = table.dataset().unwrap().get().await.unwrap();
-        let fragment_ids = dataset
-            .get_fragments()
-            .iter()
-            .map(|fragment| fragment.id())
-            .collect::<Vec<_>>();
-        assert!(fragment_ids.windows(2).all(|ids| ids[0] < ids[1]));
-
-        // A second compaction exposed the original out-of-order row-id bug.
-        table
-            .optimize(OptimizeAction::Compact {
-                options: CompactionOptions {
-                    target_rows_per_fragment: 1_000,
-                    ..Default::default()
-                },
-                remap_options: None,
-            })
-            .await
-            .unwrap();
-        assert_eq!(
-            table.count_rows(None).await.unwrap(),
-            ROWS_PER_FRAGMENT as usize * (NUM_FRAGMENTS * 2 + 1)
-        );
-    }
-
    #[tokio::test]
    async fn test_optimize_prune_versions() {
        let conn = connect("memory://").execute().await.unwrap();
@@ -537,58 +442,6 @@ mod tests {
        assert_eq!(final_row_count, 200);
    }

-    #[tokio::test]
-    async fn test_optimize_vector_index_after_delete_with_stable_row_ids() {
-        const NUM_ROWS: i32 = 400;
-        const DIMENSION: i32 = 32;
-
-        let conn = connect("memory://").execute().await.unwrap();
-        let vectors = FixedSizeListArray::try_new_from_values(
-            Float32Array::from_iter_values((0..NUM_ROWS).flat_map(|id| {
-                (0..DIMENSION).map(move |offset| ((id as f32 * 0.1) + (offset as f32 * 0.3)).sin())
-            })),
-            DIMENSION,
-        )
-        .unwrap();
-        let schema = Arc::new(Schema::new(vec![
-            Field::new("id", DataType::Int32, false),
-            Field::new("vector", vectors.data_type().clone(), false),
-        ]));
-        let batch = RecordBatch::try_new(
-            schema,
-            vec![
-                Arc::new(Int32Array::from_iter_values(0..NUM_ROWS)),
-                Arc::new(vectors),
-            ],
-        )
-        .unwrap();
-        let table = conn
-            .create_table("test_vector_index_optimize_after_delete", batch)
-            .storage_option(OPT_NEW_TABLE_ENABLE_STABLE_ROW_IDS, "true")
-            .execute()
-            .await
-            .unwrap();
-
-        table
-            .create_index(
-                &["vector"],
-                Index::IvfRq(IvfRqIndexBuilder::default().num_partitions(4)),
-            )
-            .execute()
-            .await
-            .unwrap();
-        table.delete("id % 3 = 0").await.unwrap();
-
-        // Regression test for #3330: deleted stable row IDs used to become
-        // misaligned with row addresses while joining small IVF partitions.
-        table
-            .optimize(OptimizeAction::Index(Default::default()))
-            .await
-            .unwrap();
-
-        assert_eq!(table.count_rows(None).await.unwrap(), 266);
-    }
-
    #[tokio::test]
    async fn test_optimize_all() {
        let conn = connect("memory://").execute().await.unwrap();
@@ -84,9 +84,8 @@ pub(super) async fn create_lsm_plan(
    let pk_columns = pk_columns(&ds_ref)?;
    // The base index an indexed arm relies on may lag compaction; resolve it so the
    // snapshot retains SSTables the index has not yet caught up to.
-    let arm_index = arm_maintained_index_name(&ds_ref, &query, &details).await?;
-    let (snapshots, in_memory) =
-        build_read_context(table, &ds_ref, &details, arm_index.as_deref()).await?;
+    let arm_indexes = arm_maintained_index_names(&ds_ref, &query, &details).await?;
+    let (snapshots, in_memory) = build_read_context(table, &ds_ref, &details, &arm_indexes).await?;

    let limit = query.base.limit;
    let offset = query.base.offset;
@@ -232,28 +231,40 @@ fn pk_columns(dataset: &Dataset) -> Result<Vec<String>> {
    Ok(pk)
 }

-/// Per-shard SSTable exclusion watermark: the generation at or below which SSTables
-/// are safe to drop for this arm. A generation is droppable only once it is
-/// compacted into the base table AND covered by `index_name`'s catch-up (for an
-/// indexed arm); a plain scan (`index_name == None`) uses the compaction watermark
-/// alone. Capping at the index catch-up keeps rows the base index has not yet
-/// indexed visible through their SSTable. First occurrence per shard mirrors Lance's
-/// `compacted_generation_for_shard`.
+/// Per-shard SSTable exclusion watermark: the generation at or below which
+/// SSTables are safe to drop for this query.
+///
+/// A generation is droppable only once it is compacted into the base table AND
+/// covered by the catch-up of every index the query relies on, so the watermark
+/// is the minimum across `index_names`. Gating on fewer than all of them would
+/// drop SSTables holding rows an uncounted index has not yet indexed, and that
+/// arm would silently return fewer rows.
+///
+/// See [`arm_maintained_index_names`] for which indexes are collected today: a
+/// vector search with a scalar prefilter is not yet among them.
+///
+/// An empty `index_names` (a plain scan) uses the compaction watermark alone.
+/// First occurrence per shard mirrors Lance's `compacted_generation_for_shard`.
 fn exclusion_watermarks(
    details: &MemWalIndexDetails,
-    index_name: Option<&str>,
+    index_names: &[String],
 ) -> HashMap<Uuid, u64> {
    let mut exclude: HashMap<Uuid, u64> = HashMap::new();
    for entry in &details.compacted_sstables {
        let mut watermark = entry.generation;
-        if let Some(name) = index_name
-            && let Some(caught_up) = details
+        for name in index_names {
+            match details
                .index_catchup
                .iter()
-                .find(|icp| icp.index_name == name)
+                .find(|icp| icp.index_name == *name)
                .and_then(|icp| icp.caught_up_generation_for_shard(&entry.shard_id))
-        {
-            watermark = watermark.min(caught_up);
+            {
+                Some(caught_up) => watermark = watermark.min(caught_up),
+                // No entry means the index is *not* known to hold these rows,
+                // and the base arm is index-only -- so every generation stays
+                // readable from its SSTable.
+                None => watermark = 0,
+            }
        }
        exclude.entry(entry.shard_id).or_insert(watermark);
    }
@@ -271,9 +282,9 @@ async fn build_read_context(
    table: &NativeTable,
    dataset: &Dataset,
    details: &MemWalIndexDetails,
-    index_name: Option<&str>,
+    index_names: &[String],
 ) -> Result<(Vec<ShardSnapshot>, HashMap<Uuid, InMemoryMemTables>)> {
-    let exclude = exclusion_watermarks(details, index_name);
+    let exclude = exclusion_watermarks(details, index_names);

    let shard_ids = dataset.list_mem_wal_latest_shard_ids().await?;
    // Use the dataset's own object store (not `ObjectStore::from_uri`, which
@@ -487,19 +498,33 @@ async fn index_maintained(
    }))
 }

-/// The maintained base index the query's arm relies on (vector index for ANN, FTS
-/// index for full-text), used to gate SSTable compaction exclusion by index catch-up.
-/// `None` for a plain scan or when no maintained index covers the searched column.
-async fn arm_maintained_index_name(
+/// Every maintained base index this query relies on, used to gate SSTable
+/// exclusion by index catch-up.
+///
+/// Returns a list because the watermark must be the lowest across every index a
+/// query relies on. Today it never holds more than one: `reject_unsupported`
+/// refuses hybrid search, so the vector and full-text arms are mutually
+/// exclusive.
+///
+/// The case that is genuinely multi-index -- a vector search with a scalar or
+/// bitmap prefilter -- is **not collected yet**. Identifying those needs the
+/// planner's chosen indexes, not the columns the filter names, and no Lance API
+/// exposes them. Until it does, such a query is gated on its vector index alone.
+///
+/// Empty for a plain scan, or when no maintained index covers the searched
+/// column.
+async fn arm_maintained_index_names(
    dataset: &Dataset,
    query: &VectorQueryRequest,
    details: &MemWalIndexDetails,
-) -> Result<Option<String>> {
+) -> Result<Vec<String>> {
    use lance::index::DatasetIndexExt;
-    // Resolve the arm's searched column, the index-detail type it relies on, and a
+
+    // Each arm's searched column, the index-detail type it relies on, and a
    // label for diagnostics — catch-up is taken from the vector/FTS index
    // specifically, not a BTree on the same column.
-    let (column, type_url_suffix, arm) = if !query.query_vector.is_empty() {
+    let mut arms: Vec<(String, &str, &str)> = Vec::new();
+    if !query.query_vector.is_empty() {
        let arrow_schema = ArrowSchema::from(dataset.schema());
        let column = match &query.column {
            Some(column) => column.clone(),
@@ -508,31 +533,43 @@ async fn arm_maintained_index_name(
                default_vector_column(&arrow_schema, dim)?
            }
        };
-        (column, "VectorIndexDetails", "vector")
-    } else if let Some(fts) = &query.base.full_text_search {
-        match fts.columns().into_iter().next() {
-            Some(column) => (column, "InvertedIndexDetails", "full-text"),
-            None => return Ok(None),
-        }
-    } else {
-        return Ok(None);
-    };
-    let Some(field) = dataset.schema().field(&column) else {
-        return Ok(None);
-    };
+        arms.push((column, "VectorIndexDetails", "vector"));
+    }
+    if let Some(fts) = &query.base.full_text_search
+        && let Some(column) = fts.columns().into_iter().next()
+    {
+        arms.push((column, "InvertedIndexDetails", "full-text"));
+    }
+    if arms.is_empty() {
+        return Ok(Vec::new());
+    }
+
    let indices = dataset.load_indices().await?;
-    let segment_names: Vec<String> = indices
-        .iter()
-        .filter(|idx| {
-            idx.fields.contains(&field.id)
-                && idx
-                    .index_details
-                    .as_ref()
-                    .is_some_and(|d| d.type_url.ends_with(type_url_suffix))
-        })
-        .map(|idx| idx.name.clone())
-        .collect();
-    resolve_single_index(segment_names, &details.maintained_indexes, arm, &column)
+    let mut names = Vec::with_capacity(arms.len());
+    for (column, type_url_suffix, arm) in arms {
+        let Some(field) = dataset.schema().field(&column) else {
+            continue;
+        };
+        let segment_names: Vec<String> = indices
+            .iter()
+            .filter(|idx| {
+                idx.fields.contains(&field.id)
+                    && idx
+                        .index_details
+                        .as_ref()
+                        .is_some_and(|d| d.type_url.ends_with(type_url_suffix))
+            })
+            .map(|idx| idx.name.clone())
+            .collect();
+        if let Some(name) =
+            resolve_single_index(segment_names, &details.maintained_indexes, arm, &column)?
+        {
+            names.push(name);
+        }
+    }
+    names.sort();
+    names.dedup();
+    Ok(names)
 }

 /// Resolve the single logical index from the names of its matching physical
@@ -734,24 +771,99 @@ mod tests {
        };

        // Plain scan: drop every compacted generation (through 5).
-        assert_eq!(exclusion_watermarks(&details, None).get(&shard), Some(&5));
+        assert_eq!(exclusion_watermarks(&details, &[]).get(&shard), Some(&5));

        // FTS arm with a lagging index: exclusion is capped at the index catch-up
        // (2), so SSTable generations 3..=5 are retained until the index covers
        // them — otherwise those documents would silently vanish from FTS results.
        assert_eq!(
-            exclusion_watermarks(&details, Some("fts_idx")).get(&shard),
+            exclusion_watermarks(&details, &["fts_idx".to_string()]).get(&shard),
            Some(&2)
        );

-        // A caught-up index — or one untracked in index_catchup — falls back to the
-        // compaction watermark.
+        // An index with no entry has not recorded that it holds these rows, so
+        // nothing is excluded. This is the case a table written before catch-up
+        // was maintained lands in, and it errs toward reading the SSTables.
        assert_eq!(
-            exclusion_watermarks(&details, Some("caught_up_idx")).get(&shard),
+            exclusion_watermarks(&details, &["untracked_idx".to_string()]).get(&shard),
+            Some(&0)
+        );
+
+        // An index recorded as covering the compaction watermark excludes up to it.
+        let caught_up = MemWalIndexDetails {
+            index_catchup: vec![IndexCatchupProgress::new(
+                "caught_up_idx".to_string(),
+                vec![CompactedSsTable::new(shard, 5)],
+            )],
+            ..details.clone()
+        };
+        assert_eq!(
+            exclusion_watermarks(&caught_up, &["caught_up_idx".to_string()]).get(&shard),
            Some(&5)
        );
    }

+    /// A hybrid search reads a vector and a full-text index, and either may lag.
+    /// Retaining to the lower of the two is what keeps both arms complete;
+    /// gating on one alone would drop SSTables the other has not indexed.
+    #[test]
+    fn exclusion_watermark_takes_the_minimum_across_every_index_used() {
+        let shard = Uuid::from_u128(1);
+        let details = MemWalIndexDetails {
+            compacted_sstables: vec![CompactedSsTable::new(shard, 9)],
+            index_catchup: vec![
+                IndexCatchupProgress::new(
+                    "vec_idx".to_string(),
+                    vec![CompactedSsTable::new(shard, 7)],
+                ),
+                IndexCatchupProgress::new(
+                    "fts_idx".to_string(),
+                    vec![CompactedSsTable::new(shard, 4)],
+                ),
+            ],
+            maintained_indexes: vec!["vec_idx".to_string(), "fts_idx".to_string()],
+            ..Default::default()
+        };
+
+        // Each index alone stops at its own catch-up.
+        assert_eq!(
+            exclusion_watermarks(&details, &["vec_idx".to_string()]).get(&shard),
+            Some(&7)
+        );
+        assert_eq!(
+            exclusion_watermarks(&details, &["fts_idx".to_string()]).get(&shard),
+            Some(&4)
+        );
+
+        // Used together, the lower one governs regardless of order.
+        let both = ["vec_idx".to_string(), "fts_idx".to_string()];
+        assert_eq!(exclusion_watermarks(&details, &both).get(&shard), Some(&4));
+        let reversed = ["fts_idx".to_string(), "vec_idx".to_string()];
+        assert_eq!(
+            exclusion_watermarks(&details, &reversed).get(&shard),
+            Some(&4)
+        );
+    }
+
+    /// An index with no catch-up entry is not known to hold anything, so it
+    /// governs over a lagging sibling rather than the other way round.
+    #[test]
+    fn an_untracked_index_retains_everything() {
+        let shard = Uuid::from_u128(1);
+        let details = MemWalIndexDetails {
+            compacted_sstables: vec![CompactedSsTable::new(shard, 9)],
+            index_catchup: vec![IndexCatchupProgress::new(
+                "fts_idx".to_string(),
+                vec![CompactedSsTable::new(shard, 4)],
+            )],
+            maintained_indexes: vec!["fts_idx".to_string(), "untracked_idx".to_string()],
+            ..Default::default()
+        };
+
+        let both = ["fts_idx".to_string(), "untracked_idx".to_string()];
+        assert_eq!(exclusion_watermarks(&details, &both).get(&shard), Some(&0));
+    }
+
    #[test]
    fn resolve_single_index_dedupes_segments() {
        let maintained = vec!["fts_idx".to_string()];
@@ -193,12 +193,10 @@ mod tests {

        // Add a computed column
        let result = table
-            .add_columns()
-            .transform(NewColumnTransform::SqlExpressions(vec![(
-                "doubled".into(),
-                "id * 2".into(),
-            )]))
-            .execute()
+            .add_columns(
+                NewColumnTransform::SqlExpressions(vec![("doubled".into(), "id * 2".into())]),
+                None,
+            )
            .await
            .unwrap();

@@ -253,12 +251,13 @@ mod tests {

        // Add multiple columns at once
        table
-            .add_columns()
-            .transform(NewColumnTransform::SqlExpressions(vec![
-                ("y".into(), "x + 1".into()),
-                ("z".into(), "x * x".into()),
-            ]))
-            .execute()
+            .add_columns(
+                NewColumnTransform::SqlExpressions(vec![
+                    ("y".into(), "x + 1".into()),
+                    ("z".into(), "x * x".into()),
+                ]),
+                None,
+            )
            .await
            .unwrap();

@@ -284,12 +283,10 @@ mod tests {

        // Add a column with a constant value
        table
-            .add_columns()
-            .transform(NewColumnTransform::SqlExpressions(vec![(
-                "constant".into(),
-                "42".into(),
-            )]))
-            .execute()
+            .add_columns(
+                NewColumnTransform::SqlExpressions(vec![("constant".into(), "42".into())]),
+                None,
+            )
            .await
            .unwrap();

@@ -662,12 +659,10 @@ mod tests {

        // Add column increments version
        let add_result = table
-            .add_columns()
-            .transform(NewColumnTransform::SqlExpressions(vec![(
-                "c".into(),
-                "a + b".into(),
-            )]))
-            .execute()
+            .add_columns(
+                NewColumnTransform::SqlExpressions(vec![("c".into(), "a + b".into())]),
+                None,
+            )
            .await
            .unwrap();
        assert!(add_result.version > v1);
@@ -9,17 +9,14 @@ use arrow_array::{
 };
 use arrow_schema::{DataType, Field, Fields, Schema};
 use futures::TryStreamExt;
-use lance::Dataset;
-use lance_file::version::LanceFileVersion;
+use lance_encoding::version::LanceFileVersion;
 use lancedb::{
    Connection, Error, Result, Table,
    blob::{BlobRangeRequest, blob},
    connect, connect_namespace,
-    database::listing::{
-        ListingDatabaseOptions, NewTableConfig, OPT_NEW_TABLE_ENABLE_STABLE_ROW_IDS,
-    },
+    database::listing::OPT_NEW_TABLE_ENABLE_STABLE_ROW_IDS,
    query::{ExecutableQuery, QueryBase},
-    table::{AddDataMode, CompactionOptions, OptimizeAction, OptimizeStats},
+    table::{AddDataMode, CompactionOptions, OptimizeAction},
 };
 use tempfile::tempdir;

@@ -1078,252 +1075,3 @@ async fn fetch_blob_files_aligns_across_fragments_with_nulls_and_dups() -> Resul
    }
    Ok(())
 }
-
-/// Rows exercising the null/empty interleavings from
-/// <https://github.com/lancedb/lancedb/issues/3744>: a payload, a null, a valid
-/// empty value, then payloads whose descriptors a fragment rewrite used to zero.
-fn null_empty_input_batch() -> RecordBatch {
-    let owned = [
-        Some(dedicated_blob_bytes(1)),
-        None,
-        Some(Vec::new()),
-        Some(dedicated_blob_bytes(4)),
-        Some(dedicated_blob_bytes(5)),
-        Some(dedicated_blob_bytes(6)),
-    ];
-    let payloads: Vec<Option<&[u8]>> = owned.iter().map(|payload| payload.as_deref()).collect();
-    binary_input_batch(&[1, 2, 3, 4, 5, 6], &payloads)
-}
-
-/// One `(id, Some((payload length, first byte)))` per live row, or `(id, None)`
-/// for a null blob.  Comparing lengths and first bytes keeps failure output
-/// readable where comparing whole payloads would not.
-type BlobSummary = Vec<(i64, Option<(usize, Option<u8>)>)>;
-
-/// The rows [`null_empty_input_batch`] leaves behind after `id IN (1, 4)` is
-/// deleted: a null, a valid empty value, and the two payloads that follow them.
-fn expected_null_empty_survivors() -> BlobSummary {
-    vec![
-        (2, None),
-        (3, Some((0, None))),
-        (5, Some((DEDICATED_BLOB_LEN, Some(5)))),
-        (6, Some((DEDICATED_BLOB_LEN, Some(6)))),
-    ]
-}
-
-/// `optimize()` only rewrites a fragment when lance's compaction planner selects
-/// it — here because the delete pushes the fragment past
-/// `materialize_deletions_threshold` (0.1 by default; these tests delete 2 of 6
-/// rows). Without this check, a planner or threshold change upstream would leave
-/// both regression tests green while no rewrite happened at all.
-fn assert_compacted(stats: &OptimizeStats) {
-    let metrics = stats
-        .compaction
-        .as_ref()
-        .expect("OptimizeAction::All runs compaction");
-    assert!(
-        metrics.fragments_removed >= 1,
-        "optimize() rewrote no fragment, so this test proves nothing: {metrics:?}"
-    );
-}
-
-fn summarize(rows: &[(i64, Option<Vec<u8>>)]) -> BlobSummary {
-    rows.iter()
-        .map(|(id, payload)| {
-            (
-                *id,
-                payload
-                    .as_ref()
-                    .map(|bytes| (bytes.len(), bytes.first().copied())),
-            )
-        })
-        .collect()
-}
-
-async fn sorted_id_rowid(table: &Table) -> Result<Vec<(i64, u64)>> {
-    let mut pairs = collect_id_rowid(table).await?;
-    pairs.sort_by_key(|(id, _)| *id);
-    Ok(pairs)
-}
-
-/// `{position, size}` descriptors of a legacy v1 blob column, keyed by `id`.
-async fn v1_blob_descriptors(table: &Table) -> Result<Vec<(i64, Option<(u64, u64)>)>> {
-    let batches = table
-        .query()
-        .execute()
-        .await?
-        .try_collect::<Vec<_>>()
-        .await?;
-    let batch = arrow_select::concat::concat_batches(&batches[0].schema(), &batches).unwrap();
-    let ids = batch
-        .column_by_name("id")
-        .unwrap()
-        .as_any()
-        .downcast_ref::<Int64Array>()
-        .unwrap();
-    let descriptors = batch
-        .column_by_name("image")
-        .unwrap()
-        .as_any()
-        .downcast_ref::<StructArray>()
-        .expect("v1 blob column reads back as a descriptor struct");
-    let position = descriptors
-        .column_by_name("position")
-        .unwrap()
-        .as_any()
-        .downcast_ref::<UInt64Array>()
-        .unwrap();
-    let size = descriptors
-        .column_by_name("size")
-        .unwrap()
-        .as_any()
-        .downcast_ref::<UInt64Array>()
-        .unwrap();
-    let mut rows: Vec<(i64, Option<(u64, u64)>)> = (0..batch.num_rows())
-        .map(|row| {
-            let descriptor =
-                (!descriptors.is_null(row)).then(|| (position.value(row), size.value(row)));
-            (ids.value(row), descriptor)
-        })
-        .collect();
-    rows.sort_by_key(|(id, _)| *id);
-    Ok(rows)
-}
-
-/// Payload bytes of every live row of a legacy v1 blob column, keyed by `id`.
-/// [`Table::fetch_blobs`] rejects v1 columns, so read them through lance.
-async fn v1_blob_payloads(dataset_uri: &str, table: &Table) -> Result<Vec<(i64, Option<Vec<u8>>)>> {
-    let pairs = sorted_id_rowid(table).await?;
-    let row_ids: Vec<u64> = pairs.iter().map(|(_, row_id)| *row_id).collect();
-    let dataset = Arc::new(Dataset::open(dataset_uri).await?);
-    let files = dataset.take_blobs(&row_ids, "image").await?;
-    assert_eq!(
-        files.len(),
-        pairs.len(),
-        "take_blobs returned {} handles for {} live rows",
-        files.len(),
-        pairs.len()
-    );
-    let mut rows = Vec::with_capacity(pairs.len());
-    for ((id, _), file) in pairs.iter().zip(files) {
-        let payload = match file {
-            Some(file) => Some(file.read().await?.to_vec()),
-            None => None,
-        };
-        rows.push((*id, payload));
-    }
-    Ok(rows)
-}
-
-/// Length and first byte of every live blob v2 value, keyed by `id`.
-async fn blob_v2_values(table: &Table) -> Result<BlobSummary> {
-    let pairs = sorted_id_rowid(table).await?;
-    let row_ids: Vec<u64> = pairs.iter().map(|(_, row_id)| *row_id).collect();
-    let bytes = table.fetch_blobs("image", &row_ids).await?;
-    Ok(pairs
-        .iter()
-        .enumerate()
-        .map(|(slot, (id, _))| {
-            let value = (!bytes.is_null(slot))
-                .then(|| (bytes.value(slot).len(), bytes.value(slot).first().copied()));
-            (*id, value)
-        })
-        .collect())
-}
-
-/// Regression test for [#3744]: on storage 2.0 (legacy v1 descriptors),
-/// compaction rewrote every payload following a null or empty value in the same
-/// fragment as `{position: 0, size: 0}`, so the payload bytes read back as `b""`
-/// and the new fragment no longer referenced them at all.
-///
-/// [#3744]: https://github.com/lancedb/lancedb/issues/3744
-#[tokio::test]
-async fn optimize_preserves_v1_blob_payloads_with_null_and_empty() -> Result<()> {
-    let tmp = tempdir().unwrap();
-    let db_uri = tmp.path().to_str().unwrap().to_string();
-    let db = connect(&db_uri)
-        .database_options(&ListingDatabaseOptions {
-            new_table_config: NewTableConfig {
-                data_storage_version: Some(LanceFileVersion::V2_0),
-                ..Default::default()
-            },
-            ..Default::default()
-        })
-        .execute()
-        .await?;
-    let legacy = Field::new("image", DataType::LargeBinary, true).with_metadata(
-        std::collections::HashMap::from([("lance-encoding:blob".to_string(), "true".to_string())]),
-    );
-    let schema = Arc::new(Schema::new(vec![
-        Field::new("id", DataType::Int64, false),
-        legacy,
-    ]));
-    let table = db.create_empty_table("t", schema).execute().await?;
-    table.add(null_empty_input_batch()).execute().await?;
-    assert_eq!(
-        storage_format_version(&table).await,
-        LanceFileVersion::V2_0.resolve(),
-        "v1 blob descriptors only exist below storage 2.2"
-    );
-    let dataset_uri = table.uri().await?;
-
-    // Any rewrite triggers it; deleting rows is the shape from the issue.
-    table.delete("id IN (1, 4)").await?;
-    let descriptors_before = v1_blob_descriptors(&table).await?;
-    let before = v1_blob_payloads(&dataset_uri, &table).await?;
-    assert_eq!(
-        summarize(&before),
-        expected_null_empty_survivors(),
-        "test setup no longer produces the null/empty/payload mix"
-    );
-
-    let stats = table.optimize(OptimizeAction::All).await?;
-    assert_compacted(&stats);
-
-    let descriptors_after = v1_blob_descriptors(&table).await?;
-    let after = v1_blob_payloads(&dataset_uri, &table).await?;
-    assert_eq!(
-        summarize(&after),
-        summarize(&before),
-        "optimize() lost blob payloads; descriptors before={descriptors_before:?} after={descriptors_after:?}"
-    );
-    assert!(after == before, "optimize() changed blob payload bytes");
-    Ok(())
-}
-
-/// Regression test for the blob v2 half of [#3744]: compaction rewrote a valid
-/// empty value as null, destroying the null-vs-empty distinction.
-///
-/// [#3744]: https://github.com/lancedb/lancedb/issues/3744
-#[tokio::test]
-async fn optimize_preserves_blob_v2_null_and_empty_distinction() -> Result<()> {
-    let tmp = tempdir().unwrap();
-    let db = connect(tmp.path().to_str().unwrap()).execute().await?;
-    let table = db
-        .create_empty_table("t", blob_table_schema())
-        .execute()
-        .await?;
-    table.add(null_empty_input_batch()).execute().await?;
-    assert!(
-        storage_format_version(&table).await >= LanceFileVersion::V2_2,
-        "blob v2 columns require storage >= 2.2"
-    );
-
-    table.delete("id IN (1, 4)").await?;
-    let before = blob_v2_values(&table).await?;
-    assert_eq!(
-        before,
-        expected_null_empty_survivors(),
-        "test setup no longer produces the null/empty/payload mix"
-    );
-
-    let stats = table.optimize(OptimizeAction::All).await?;
-    assert_compacted(&stats);
-
-    assert_eq!(
-        blob_v2_values(&table).await?,
-        before,
-        "optimize() changed blob v2 values"
-    );
-    Ok(())
-}
Author	SHA1	Message	Date
XYZhan	6c4269bbb0	refactor(lsm): remove the index-catchup activation surface Follows the lance change that removes the feature bit. With one set of semantics there is no mode to switch into, so `require_mem_wal_index_catchup` goes from the trait, `NativeTable`, and the LSM merge module. `exclusion_watermarks` loses its `catchup_required` argument and keeps the conservative branch: an index with no `index_catchup` entry is not known to hold the compacted rows, so its generations stay readable from their SSTables. That is what makes an existing table safe to read the moment the new binary starts -- nothing is excluded until an index records that it covers it. Two tests changed because they encoded the branch that is gone, and they had conflated two different things: an index that is caught up and an index that is untracked both fell back to the compaction watermark. Untracked now retains everything, so the tests assert that and cover the genuinely-caught-up case separately. 484 lancedb lib tests pass.	2026-08-20 18:29:56 -04:00
XYZhan	41e0161067	feat(lsm): move catch-up activation to an explicit API	2026-08-10 09:10:05 -04:00
XYZhan	f75279b69f	feat(lsm): strict catch-up semantics and activation, on the 10.1 line lancedb#3780 plus the two pieces it deferred: a missing index_catchup entry means not-caught-up once the feature bit is set, and turning WAL on requires catch-up while the table is provably clean. Local only, for end-to-end work before the lance bump.	2026-08-09 23:19:21 -04:00
lancedb automation	667cf32e78	chore: update lance dependency to v10.1.0-beta.2	2026-08-02 00:12:47 +00:00