refactor!: rename MemWAL APIs to RTE (realtime engine) (#4598)

## What

Renames the user-facing MemWAL surface to **RTE** (realtime engine)
across Rust, Python, TypeScript, and Java, including the REST wire
format. This follows the LSM → MemWAL rename in #4075.

| Before | After |
|---|---|
| `MemWalWriteSpec` / `MemWalStats` | `RTEWriteSpec` / `RTEStats` |
| `enable_memwal` / `disable_memwal` / `get_memwal_spec` | `enable_rte`
/ `disable_rte` / `get_rte_spec` |
| `flush_memwal` / `compact_memwal` / `checkpoint_memwal` | `flush_rte`
/ `compact_rte` / `checkpoint_rte` |
| `get_memwal_stats` / `close_memwal_writers` | `get_rte_stats` /
`close_rte_writers` |
| `use_memwal` (query, merge insert) | `use_rte` |
| TS/Java `enableMemWal`, `useMemWal`, ... | `enableRTE`, `useRTE`, ...
(napi `js_name` set explicitly) |
| Java `LanceDbTableMemWal` | `LanceDbTableRTE` |
| `table/memwal_stats.rs` | `table/rte_stats.rs` |

Docs and comments now say "realtime engine (RTE)". Text about Lance's
underlying mechanism keeps "MemWAL" ("Lance's MemWAL", "MemWAL index").

**Left alone:** Lance-owned names (`MemWalIndexDetails`,
`DatasetMemWalExt`, `InitializeMemWal*`, `mem_wal` paths), internal
`Lsm*` names and modules, and the `does not support with_row_id` refusal
text that the hybrid fallback matches on.

## Compatibility

Breaking, for an experimental API. The wire format changes, so this must
roll out together with the matching sophon change (lancedb/sophon#8870):

- Routes: `/enable_memwal/` → `/enable_rte/`, `/disable_memwal/` →
`/disable_rte/`, `/get_memwal_spec/` → `/get_rte_spec/`,
`/get_memwal_stats/` → `/get_rte_stats/`, `/flush_memwal/` →
`/flush_rte/`, `/compact_memwal/` → `/compact_rte/`
- Response keys: `memwal_spec` → `rte_spec`, `memwal_stats` →
`rte_stats`
- Request field: `use_memwal` → `use_rte` on query and merge-insert
bodies

The SQL keyword becomes `ALTER TABLE ... SET RTE` server-side. The
guides under `docs/web` will be updated separately.

## Validation

- Rust: `cargo clippy --features remote --tests --examples` clean;
`cargo test --features remote -p lancedb --lib`: 1533 passed
- Java: `mvn test`: 37 passed (Spotless applied with JDK 11)
- Python: `ruff format` / `ruff check` clean; targeted tests
(`test_rte_spec`, `test_merge_insert_lsm`, `test_remote_db`,
`test_wal_hybrid`, `test_query`, `test_permutation`, `test_table`): 708
passed
- TypeScript: `pnpm build`; `table`, `query` tests: 606 passed; `biome
check` clean; `docs/src/js` regenerated with `pnpm run docs`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dan Rammer
2026-10-20 11:58:57 -05:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 1772f4ef83
commit 92825f6f13
73 changed files with 1656 additions and 1600 deletions
+14 -14
View File
@@ -333,7 +333,7 @@ pub struct PyQueryRequest {
pub select_source_columns: Option<HashMap<String, String>>,
pub fast_search: Option<bool>,
pub with_row_id: Option<bool>,
pub use_memwal: Option<bool>,
pub use_rte: Option<bool>,
pub column: Option<String>,
pub query_vector: Option<PyQueryVectors>,
pub nprobes: Option<usize>,
@@ -366,7 +366,7 @@ impl From<AnyQuery> for PyQueryRequest {
select: PySelect(query_request.select),
fast_search: Some(query_request.fast_search),
with_row_id: Some(query_request.with_row_id),
use_memwal: query_request.use_memwal,
use_rte: query_request.use_rte,
column: None,
query_vector: None,
nprobes: None,
@@ -395,7 +395,7 @@ impl From<AnyQuery> for PyQueryRequest {
select: PySelect(vector_query.base.select),
fast_search: Some(vector_query.base.fast_search),
with_row_id: Some(vector_query.base.with_row_id),
use_memwal: vector_query.base.use_memwal,
use_rte: vector_query.base.use_rte,
column: vector_query.column,
query_vector: Some(PyQueryVectors(vector_query.query_vector)),
nprobes: vector_query.nprobes,
@@ -540,8 +540,8 @@ impl Query {
self.inner = self.inner.clone().fast_search();
}
pub fn use_memwal(&mut self, enable: bool) {
self.inner = self.inner.clone().use_memwal(enable);
pub fn use_rte(&mut self, enable: bool) {
self.inner = self.inner.clone().use_rte(enable);
}
pub fn with_row_id(&mut self) {
@@ -706,8 +706,8 @@ impl TakeQuery {
self.inner = self.inner.clone().with_row_id();
}
pub fn use_memwal(&mut self, enable: bool) {
self.inner = self.inner.clone().use_memwal(enable);
pub fn use_rte(&mut self, enable: bool) {
self.inner = self.inner.clone().use_rte(enable);
}
#[pyo3(signature = ())]
@@ -819,8 +819,8 @@ impl FTSQuery {
self.inner = self.inner.clone().fast_search();
}
pub fn use_memwal(&mut self, enable: bool) {
self.inner = self.inner.clone().use_memwal(enable);
pub fn use_rte(&mut self, enable: bool) {
self.inner = self.inner.clone().use_rte(enable);
}
pub fn with_row_id(&mut self) {
@@ -970,8 +970,8 @@ impl VectorQuery {
self.inner = self.inner.clone().fast_search();
}
pub fn use_memwal(&mut self, enable: bool) {
self.inner = self.inner.clone().use_memwal(enable);
pub fn use_rte(&mut self, enable: bool) {
self.inner = self.inner.clone().use_rte(enable);
}
pub fn with_row_id(&mut self) {
@@ -1178,9 +1178,9 @@ impl HybridQuery {
self.inner_fts.postfilter();
}
pub fn use_memwal(&mut self, enable: bool) {
self.inner_vec.use_memwal(enable);
self.inner_fts.use_memwal(enable);
pub fn use_rte(&mut self, enable: bool) {
self.inner_vec.use_rte(enable);
self.inner_fts.use_rte(enable);
}
pub fn add_query_vector(&mut self, vector: Bound<'_, PyAny>) -> PyResult<()> {