mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-21 19:22:27 +00:00
## What Renames the user-facing MemWAL surface to **RTE** (realtime engine) across Rust, Python, TypeScript, and Java, including the REST wire format. This follows the LSM → MemWAL rename in #4075. | Before | After | |---|---| | `MemWalWriteSpec` / `MemWalStats` | `RTEWriteSpec` / `RTEStats` | | `enable_memwal` / `disable_memwal` / `get_memwal_spec` | `enable_rte` / `disable_rte` / `get_rte_spec` | | `flush_memwal` / `compact_memwal` / `checkpoint_memwal` | `flush_rte` / `compact_rte` / `checkpoint_rte` | | `get_memwal_stats` / `close_memwal_writers` | `get_rte_stats` / `close_rte_writers` | | `use_memwal` (query, merge insert) | `use_rte` | | TS/Java `enableMemWal`, `useMemWal`, ... | `enableRTE`, `useRTE`, ... (napi `js_name` set explicitly) | | Java `LanceDbTableMemWal` | `LanceDbTableRTE` | | `table/memwal_stats.rs` | `table/rte_stats.rs` | Docs and comments now say "realtime engine (RTE)". Text about Lance's underlying mechanism keeps "MemWAL" ("Lance's MemWAL", "MemWAL index"). **Left alone:** Lance-owned names (`MemWalIndexDetails`, `DatasetMemWalExt`, `InitializeMemWal*`, `mem_wal` paths), internal `Lsm*` names and modules, and the `does not support with_row_id` refusal text that the hybrid fallback matches on. ## Compatibility Breaking, for an experimental API. The wire format changes, so this must roll out together with the matching sophon change (lancedb/sophon#8870): - Routes: `/enable_memwal/` → `/enable_rte/`, `/disable_memwal/` → `/disable_rte/`, `/get_memwal_spec/` → `/get_rte_spec/`, `/get_memwal_stats/` → `/get_rte_stats/`, `/flush_memwal/` → `/flush_rte/`, `/compact_memwal/` → `/compact_rte/` - Response keys: `memwal_spec` → `rte_spec`, `memwal_stats` → `rte_stats` - Request field: `use_memwal` → `use_rte` on query and merge-insert bodies The SQL keyword becomes `ALTER TABLE ... SET RTE` server-side. The guides under `docs/web` will be updated separately. ## Validation - Rust: `cargo clippy --features remote --tests --examples` clean; `cargo test --features remote -p lancedb --lib`: 1533 passed - Java: `mvn test`: 37 passed (Spotless applied with JDK 11) - Python: `ruff format` / `ruff check` clean; targeted tests (`test_rte_spec`, `test_merge_insert_lsm`, `test_remote_db`, `test_wal_hybrid`, `test_query`, `test_permutation`, `test_table`): 708 passed - TypeScript: `pnpm build`; `table`, `query` tests: 606 passed; `biome check` clean; `docs/src/js` regenerated with `pnpm run docs` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
LanceDB Java Enterprise Client
Configuration and Initialization
LanceDB Cloud
For LanceDB Cloud, use the simplified builder API:
import com.lancedb.LanceDbNamespaceClientBuilder;
import org.lance.namespace.LanceNamespace;
// If your DB url is db://example-db, then your database here is example-db
LanceNamespace namespaceClient = LanceDbNamespaceClientBuilder.newBuilder()
.apiKey("your_lancedb_cloud_api_key")
.database("your_database_name")
.build();
LanceDB Enterprise
For Enterprise deployments, use your custom endpoint:
LanceNamespace namespaceClient = LanceDbNamespaceClientBuilder.newBuilder()
.apiKey("your_lancedb_enterprise_api_key")
.database("your_database_name")
.endpoint("<your_enterprise_endpoint>")
.build();
RTE write path
Most table operations reach LanceDB through the LanceNamespace above, which is
generated from the Lance Namespace specification. The RTE LSM routes are not part
of that specification, so they are issued through a separate client:
import com.lancedb.LanceDbRestClient;
import com.lancedb.LanceDbTableRTE;
import com.lancedb.RTEWriteSpec;
LanceDbRestClient client = LanceDbNamespaceClientBuilder.newBuilder()
.apiKey("your_lancedb_cloud_api_key")
.database("your_database_name")
.buildRestClient();
LanceDbTableRTE rte = new LanceDbTableRTE(client, "my_table");
// Route future merge_insert upserts through the RTE, hash-bucketed by `id`.
rte.enableRTE(RTEWriteSpec.bucket("id", 16));
// ... merge_insert traffic ...
// Converge the fresh tier into the base table.
rte.checkpointRTE();
// Inspect live per-table-shard state.
rte.getRTEStats().ifPresent(stats -> stats.tableShards().forEach(tableShard ->
System.out.println(tableShard.shardId() + ": " + tableShard.sstables().size() + " SSTables")));
client.close();
maintainedIndexes is tri-state, and the null default is the opposite of what a Java
reader usually expects:
| Value | Meaning |
|---|---|
| unset (null) | Maintain every index the table has, including ones created later |
Collections.emptyList() |
Maintain none |
Arrays.asList("id_idx") |
Maintain exactly those |
Development
The parent POM includes https://maven.lance.org to resolve Lance beta and RC
dependencies. Use an explicit lance-core.version; numbered beta and RC versions
are Maven releases, not snapshots. Maven Central remains available for stable
and historical versions, as well as other dependencies.
Build:
./mvnw install -pl lancedb-core -am
Run tests:
./mvnw test -pl lancedb-core