mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 23:32:27 +00:00
Daniel RammerandClaude Opus 5.5
d7e921dbec
The write path these APIs drive is Lance's MemWAL, and "LSM" described its shape rather than naming it. The surface now says MemWAL, and turning it on and off reads as enabling a feature rather than installing a spec. | Before | After | |---|---| | `set_lsm_write_spec` | `enable_memwal` | | `unset_lsm_write_spec` | `disable_memwal` | | `get_lsm_write_spec` | `get_memwal_spec` | | `flush_lsm` / `compact_lsm` / `checkpoint_lsm` | `flush_memwal` / `compact_memwal` / `checkpoint_memwal` | | `get_lsm_stats` | `get_memwal_stats` | | `close_lsm_writers` | `close_memwal_writers` | | `use_lsm` (query, merge insert) | `use_memwal` | | `LsmWriteSpec` / `LsmStats` | `MemWalWriteSpec` / `MemWalStats` | | Java `LanceDbTableLsm` | `LanceDbTableMemWal` | TypeScript and Java use camelCase (`enableMemWal`, `getMemWalStats`, ...); the napi methods set `js_name` so the generated names keep the `MemWal` casing. The flush/compact/checkpoint methods keep a `_memwal` suffix so `compact_memwal` is not confused with `compact_files` and `optimize`, which compact the base table. ## Wire The remote routes and JSON envelopes move with the SDKs and must ship together with the matching server change: - `/set_lsm_write_spec/` -> `/enable_memwal/`, `/unset_lsm_write_spec/` -> `/disable_memwal/`, `/get_lsm_write_spec/` -> `/get_memwal_spec/`, `/flush_lsm/`, `/compact_lsm/`, `/get_lsm_stats/` -> `*_memwal/` - response keys `lsm_write_spec` -> `memwal_spec`, `lsm_stats` -> `memwal_stats`; query and merge-insert body field `use_lsm` -> `use_memwal` ## Also fixed The previous rename turned the Java `bucket` sharding mode into `tableShard` in the javadoc example and tests, and the test fixtures sent `tableShards` where the parser reads `table_shards`, so the Java tests did not compile. Both are restored. Internal names (`LsmMode`, `LsmScanner`, the `merge::lsm` and `query::lsm` modules) are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
92 lines
2.6 KiB
Markdown
92 lines
2.6 KiB
Markdown
# LanceDB Java Enterprise Client
|
|
|
|
## Configuration and Initialization
|
|
|
|
### LanceDB Cloud
|
|
|
|
For LanceDB Cloud, use the simplified builder API:
|
|
|
|
```java
|
|
import com.lancedb.LanceDbNamespaceClientBuilder;
|
|
import org.lance.namespace.LanceNamespace;
|
|
|
|
// If your DB url is db://example-db, then your database here is example-db
|
|
LanceNamespace namespaceClient = LanceDbNamespaceClientBuilder.newBuilder()
|
|
.apiKey("your_lancedb_cloud_api_key")
|
|
.database("your_database_name")
|
|
.build();
|
|
```
|
|
|
|
### LanceDB Enterprise
|
|
|
|
For Enterprise deployments, use your custom endpoint:
|
|
|
|
```java
|
|
LanceNamespace namespaceClient = LanceDbNamespaceClientBuilder.newBuilder()
|
|
.apiKey("your_lancedb_enterprise_api_key")
|
|
.database("your_database_name")
|
|
.endpoint("<your_enterprise_endpoint>")
|
|
.build();
|
|
```
|
|
|
|
## MemWAL write path
|
|
|
|
Most table operations reach LanceDB through the `LanceNamespace` above, which is
|
|
generated from the Lance Namespace specification. The MemWAL LSM routes are not part
|
|
of that specification, so they are issued through a separate client:
|
|
|
|
```java
|
|
import com.lancedb.LanceDbRestClient;
|
|
import com.lancedb.LanceDbTableMemWal;
|
|
import com.lancedb.MemWalWriteSpec;
|
|
|
|
LanceDbRestClient client = LanceDbNamespaceClientBuilder.newBuilder()
|
|
.apiKey("your_lancedb_cloud_api_key")
|
|
.database("your_database_name")
|
|
.buildRestClient();
|
|
|
|
LanceDbTableMemWal memWal = new LanceDbTableMemWal(client, "my_table");
|
|
|
|
// Route future merge_insert upserts through the MemWAL, hash-bucketed by `id`.
|
|
memWal.enableMemWal(MemWalWriteSpec.bucket("id", 16));
|
|
|
|
// ... merge_insert traffic ...
|
|
|
|
// Converge the fresh tier into the base table.
|
|
memWal.checkpointMemWal();
|
|
|
|
// Inspect live per-table-shard state.
|
|
memWal.getMemWalStats().ifPresent(stats -> stats.tableShards().forEach(tableShard ->
|
|
System.out.println(tableShard.shardId() + ": " + tableShard.sstables().size() + " SSTables")));
|
|
|
|
client.close();
|
|
```
|
|
|
|
`maintainedIndexes` is tri-state, and the null default is the opposite of what a Java
|
|
reader usually expects:
|
|
|
|
| Value | Meaning |
|
|
| --- | --- |
|
|
| unset (null) | Maintain **every** index the table has, including ones created later |
|
|
| `Collections.emptyList()` | Maintain **none** |
|
|
| `Arrays.asList("id_idx")` | Maintain exactly those |
|
|
|
|
## Development
|
|
|
|
The parent POM includes `https://maven.lance.org` to resolve Lance beta and RC
|
|
dependencies. Use an explicit `lance-core.version`; numbered beta and RC versions
|
|
are Maven releases, not snapshots. Maven Central remains available for stable
|
|
and historical versions, as well as other dependencies.
|
|
|
|
Build:
|
|
|
|
```shell
|
|
./mvnw install -pl lancedb-core -am
|
|
```
|
|
|
|
Run tests:
|
|
|
|
```shell
|
|
./mvnw test -pl lancedb-core
|
|
```
|