The write path these APIs drive is Lance's MemWAL, and "LSM" described
its shape rather than naming it. The surface now says MemWAL, and turning
it on and off reads as enabling a feature rather than installing a spec.
| Before | After |
|---|---|
| `set_lsm_write_spec` | `enable_memwal` |
| `unset_lsm_write_spec` | `disable_memwal` |
| `get_lsm_write_spec` | `get_memwal_spec` |
| `flush_lsm` / `compact_lsm` / `checkpoint_lsm` | `flush_memwal` / `compact_memwal` / `checkpoint_memwal` |
| `get_lsm_stats` | `get_memwal_stats` |
| `close_lsm_writers` | `close_memwal_writers` |
| `use_lsm` (query, merge insert) | `use_memwal` |
| `LsmWriteSpec` / `LsmStats` | `MemWalWriteSpec` / `MemWalStats` |
| Java `LanceDbTableLsm` | `LanceDbTableMemWal` |
TypeScript and Java use camelCase (`enableMemWal`, `getMemWalStats`,
...); the napi methods set `js_name` so the generated names keep the
`MemWal` casing. The flush/compact/checkpoint methods keep a `_memwal`
suffix so `compact_memwal` is not confused with `compact_files` and
`optimize`, which compact the base table.
## Wire
The remote routes and JSON envelopes move with the SDKs and must ship
together with the matching server change:
- `/set_lsm_write_spec/` -> `/enable_memwal/`, `/unset_lsm_write_spec/`
-> `/disable_memwal/`, `/get_lsm_write_spec/` -> `/get_memwal_spec/`,
`/flush_lsm/`, `/compact_lsm/`, `/get_lsm_stats/` -> `*_memwal/`
- response keys `lsm_write_spec` -> `memwal_spec`, `lsm_stats` ->
`memwal_stats`; query and merge-insert body field `use_lsm` ->
`use_memwal`
## Also fixed
The previous rename turned the Java `bucket` sharding mode into
`tableShard` in the javadoc example and tests, and the test fixtures
sent `tableShards` where the parser reads `table_shards`, so the Java
tests did not compile. Both are restored.
Internal names (`LsmMode`, `LsmScanner`, the `merge::lsm` and
`query::lsm` modules) are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Applies the SSTable / table shard / freeze vocabulary from the previous
commit to code that came in from main, and to spots that commit missed:
- `get_lsm_stats` test fixtures in Rust and Python still sent the old
`buckets` / `generations` keys, so the Rust test target no longer
compiled and the Python tests would fail against `table_shards`.
- The Java javadoc example called a nonexistent `LsmWriteSpec.tableShard`,
an over-rename of `LsmWriteSpec.bucket`; the README used the old
`buckets()` / `generations()` accessors.
- Prose that still said bucket (for a shard), seal, L0, or flushed
generations, and identifier-style "table_shard" / "tableShard" in prose.
`LsmWriteSpec::bucket`, `num_buckets`, the hash-bucket values in the
merge-insert router, and the `"Sealed"` status wire value are unchanged:
those name the sharding transform or a server value, not the shard.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
## Summary
The Java SDK pinned `jackson-databind` 2.17.1, and `jackson-core` and
the other Jackson modules came in transitively via `arrow-vector` and
`lance-namespace-apache-client` at 2.16.x–2.17.x, all affected by known
CVEs.
This imports `jackson-bom` 2.18.11 in the parent POM, and declares
`jackson-core`, `jackson-annotations`, `jackson-datatype-jsr310`, and
`jackson-jaxrs-json-provider` as direct (BOM-managed) dependencies of
`lancedb-core`. The direct declarations are needed because the parent's
dependency management doesn't apply to a consumer's transitive
resolution; without them, Maven's nearest-wins mediation still hands
downstream apps the older versions from `arrow-vector`.
## Evidence
`mvn dependency:list` in a separate consumer project depending only on
`lancedb-core`:
| Artifact | Before | After |
|---|---|---|
| jackson-core | 2.16.0 | 2.18.11 |
| jackson-annotations | 2.16.0 | 2.18.11 |
| jackson-databind | 2.17.1 | 2.18.11 |
| jackson-datatype-jsr310 | 2.16.0 | 2.18.11 |
| jackson-jaxrs-base / json-provider | 2.17.1 | 2.18.11 |
| jackson-module-jaxb-annotations | 2.17.1 | 2.18.11 |
## Merge Risk
**Door:** Two-way. Dependency versions only, and they stay within
Jackson 2.x.
**Blast Radius:** `java` dependencies. Downstream users of
`lancedb-core` now get Jackson 2.18.11 transitively.
Fixes OSS-2360, OSS-2337
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`maintained_indexes` on a MemWAL table has three values: a named list,
`[]` for none, and `null` for every index the table has, including ones
created later. This PR makes every client read all three back as they
were set.
- **Rust remote client** (also used by the Python and Node SDKs) reads
`null` as `None`.
- **Java client** reads `null` as `null`, not as an empty list.
- **Docs** for the setter, the field and the getter describe the three
values the same way in Rust, Python, Node and Java. The generated JS
docs are regenerated to match.
## Testing
- `rust/lancedb`: 73 LSM tests pass, including one case per value. With
the decoder typed as a list, the `null` case fails.
- Java: two cases for `null` and `[]` on the read path, run by the Java
CI job.
- `cargo fmt`, `cargo clippy -D warnings` and `ruff format` are clean.
Aligns the LSM stats surface with the MemWAL naming settled upstream in
lance-format/lance#7943 and #7957, where the persisted unit became an
SSTable. Two terms in this API predate that pass.
**Generation -> SSTable.** A `GenerationStats` describes one flushed
MemTable, which is an SSTable. The generation *number* is kept — an
SSTable is identified by its generation — so only the noun moved.
**Bucket -> table shard.** Each entry is one MemWAL shard. "Bucket" names
only the hash sharding transform, so it was wrong for `identity` and
`year` sharding, which produce shards and no buckets at all.
| Before | After |
|---|---|
| `GenerationStats` | `SsTableStats` |
| `BucketStats` | `TableShardStats` |
| `LsmStats.buckets` | `LsmStats.table_shards` |
| `BucketStats.generations` | `TableShardStats.sstables` |
| `include_generation_rows` | `include_sstable_rows` |
| `newest_generation` | `newest_sstable_generation` |
| `outstanding_generations` | `outstanding_sstables` |
Applied across Rust, Python, TypeScript, and Java, including the
`get_lsm_stats` JSON field names. `LsmWriteSpec::Bucket` and `num_buckets`
are unchanged — those name the sharding transform, not the shard.
## Compatibility
Breaking for the MemWAL LSM stats API, which is experimental and paired
with a server that renames the same fields. The JSON keys `table_shards`
and `sstables` must roll out together with the WAL server change.
## Validation
- `cargo check -p lancedb --all-features`, `cargo fmt --all`
- `ruff format --check` and `ruff check` on the touched Python
- `biome check` on the touched TypeScript
The typedoc markdown under `docs/src/js` was updated by hand, not
regenerated: `npm run docs` needs the napi-built `./native` types. Worth
running `npm run docs` on this branch to confirm the generator agrees.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HgVf5C2yRbwK6pe1aMYkmg
Daniel RammerandClaude Opus 5.5
Lance Release
Will JonesandClaude Opus 5.5
XY Zhan