mirror of https://github.com/lancedb/lancedb.git synced 2026-10-13 15:22:30 +00:00

Files

T

Daniel RammerandClaude Opus 5.5 d7e921dbec refactor!: rename the LSM table APIs to MemWAL

The write path these APIs drive is Lance's MemWAL, and "LSM" described
its shape rather than naming it. The surface now says MemWAL, and turning
it on and off reads as enabling a feature rather than installing a spec.

| Before | After |
|---|---|
| `set_lsm_write_spec` | `enable_memwal` |
| `unset_lsm_write_spec` | `disable_memwal` |
| `get_lsm_write_spec` | `get_memwal_spec` |
| `flush_lsm` / `compact_lsm` / `checkpoint_lsm` | `flush_memwal` / `compact_memwal` / `checkpoint_memwal` |
| `get_lsm_stats` | `get_memwal_stats` |
| `close_lsm_writers` | `close_memwal_writers` |
| `use_lsm` (query, merge insert) | `use_memwal` |
| `LsmWriteSpec` / `LsmStats` | `MemWalWriteSpec` / `MemWalStats` |
| Java `LanceDbTableLsm` | `LanceDbTableMemWal` |

TypeScript and Java use camelCase (`enableMemWal`, `getMemWalStats`,
...); the napi methods set `js_name` so the generated names keep the
`MemWal` casing. The flush/compact/checkpoint methods keep a `_memwal`
suffix so `compact_memwal` is not confused with `compact_files` and
`optimize`, which compact the base table.

## Wire

The remote routes and JSON envelopes move with the SDKs and must ship
together with the matching server change:

- `/set_lsm_write_spec/` -> `/enable_memwal/`, `/unset_lsm_write_spec/`
  -> `/disable_memwal/`, `/get_lsm_write_spec/` -> `/get_memwal_spec/`,
  `/flush_lsm/`, `/compact_lsm/`, `/get_lsm_stats/` -> `*_memwal/`
- response keys `lsm_write_spec` -> `memwal_spec`, `lsm_stats` ->
  `memwal_stats`; query and merge-insert body field `use_lsm` ->
  `use_memwal`

## Also fixed

The previous rename turned the Java `bucket` sharding mode into
`tableShard` in the javadoc example and tests, and the test fixtures
sent `tableShards` where the parser reads `table_shards`, so the Java
tests did not compile. Both are restored.

Internal names (`LsmMode`, `LsmScanner`, the `merge::lsm` and
`query::lsm` modules) are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-12 15:54:03 -05:00

2.6 KiB

Raw Permalink Blame History

LanceDB Java Enterprise Client

Configuration and Initialization

LanceDB Cloud

For LanceDB Cloud, use the simplified builder API:

import com.lancedb.LanceDbNamespaceClientBuilder;
import org.lance.namespace.LanceNamespace;

// If your DB url is db://example-db, then your database here is example-db
LanceNamespace namespaceClient = LanceDbNamespaceClientBuilder.newBuilder()
    .apiKey("your_lancedb_cloud_api_key")
    .database("your_database_name")
    .build();

LanceDB Enterprise

For Enterprise deployments, use your custom endpoint:

LanceNamespace namespaceClient = LanceDbNamespaceClientBuilder.newBuilder()
    .apiKey("your_lancedb_enterprise_api_key")
    .database("your_database_name")
    .endpoint("<your_enterprise_endpoint>")
    .build();

MemWAL write path

Most table operations reach LanceDB through the LanceNamespace above, which is generated from the Lance Namespace specification. The MemWAL LSM routes are not part of that specification, so they are issued through a separate client:

import com.lancedb.LanceDbRestClient;
import com.lancedb.LanceDbTableMemWal;
import com.lancedb.MemWalWriteSpec;

LanceDbRestClient client = LanceDbNamespaceClientBuilder.newBuilder()
    .apiKey("your_lancedb_cloud_api_key")
    .database("your_database_name")
    .buildRestClient();

LanceDbTableMemWal memWal = new LanceDbTableMemWal(client, "my_table");

// Route future merge_insert upserts through the MemWAL, hash-bucketed by `id`.
memWal.enableMemWal(MemWalWriteSpec.bucket("id", 16));

// ... merge_insert traffic ...

// Converge the fresh tier into the base table.
memWal.checkpointMemWal();

// Inspect live per-table-shard state.
memWal.getMemWalStats().ifPresent(stats -> stats.tableShards().forEach(tableShard ->
    System.out.println(tableShard.shardId() + ": " + tableShard.sstables().size() + " SSTables")));

client.close();

maintainedIndexes is tri-state, and the null default is the opposite of what a Java reader usually expects:

Value	Meaning
unset (null)	Maintain every index the table has, including ones created later
`Collections.emptyList()`	Maintain none
`Arrays.asList("id_idx")`	Maintain exactly those

Development

The parent POM includes https://maven.lance.org to resolve Lance beta and RC dependencies. Use an explicit lance-core.version; numbered beta and RC versions are Maven releases, not snapshots. Maven Central remains available for stable and historical versions, as well as other dependencies.

Build:

./mvnw install -pl lancedb-core -am

Run tests:

./mvnw test -pl lancedb-core

2.6 KiB Raw Permalink Blame History