mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-15 15:16:53 +00:00
refactor!: rename MemWAL APIs to RTE (realtime engine) (#4598)
## What Renames the user-facing MemWAL surface to **RTE** (realtime engine) across Rust, Python, TypeScript, and Java, including the REST wire format. This follows the LSM → MemWAL rename in #4075. | Before | After | |---|---| | `MemWalWriteSpec` / `MemWalStats` | `RTEWriteSpec` / `RTEStats` | | `enable_memwal` / `disable_memwal` / `get_memwal_spec` | `enable_rte` / `disable_rte` / `get_rte_spec` | | `flush_memwal` / `compact_memwal` / `checkpoint_memwal` | `flush_rte` / `compact_rte` / `checkpoint_rte` | | `get_memwal_stats` / `close_memwal_writers` | `get_rte_stats` / `close_rte_writers` | | `use_memwal` (query, merge insert) | `use_rte` | | TS/Java `enableMemWal`, `useMemWal`, ... | `enableRTE`, `useRTE`, ... (napi `js_name` set explicitly) | | Java `LanceDbTableMemWal` | `LanceDbTableRTE` | | `table/memwal_stats.rs` | `table/rte_stats.rs` | Docs and comments now say "realtime engine (RTE)". Text about Lance's underlying mechanism keeps "MemWAL" ("Lance's MemWAL", "MemWAL index"). **Left alone:** Lance-owned names (`MemWalIndexDetails`, `DatasetMemWalExt`, `InitializeMemWal*`, `mem_wal` paths), internal `Lsm*` names and modules, and the `does not support with_row_id` refusal text that the hybrid fallback matches on. ## Compatibility Breaking, for an experimental API. The wire format changes, so this must roll out together with the matching sophon change (lancedb/sophon#8870): - Routes: `/enable_memwal/` → `/enable_rte/`, `/disable_memwal/` → `/disable_rte/`, `/get_memwal_spec/` → `/get_rte_spec/`, `/get_memwal_stats/` → `/get_rte_stats/`, `/flush_memwal/` → `/flush_rte/`, `/compact_memwal/` → `/compact_rte/` - Response keys: `memwal_spec` → `rte_spec`, `memwal_stats` → `rte_stats` - Request field: `use_memwal` → `use_rte` on query and merge-insert bodies The SQL keyword becomes `ALTER TABLE ... SET RTE` server-side. The guides under `docs/web` will be updated separately. ## Validation - Rust: `cargo clippy --features remote --tests --examples` clean; `cargo test --features remote -p lancedb --lib`: 1533 passed - Java: `mvn test`: 37 passed (Spotless applied with JDK 11) - Python: `ruff format` / `ruff check` clean; targeted tests (`test_rte_spec`, `test_merge_insert_lsm`, `test_remote_db`, `test_wal_hybrid`, `test_query`, `test_permutation`, `test_table`): 708 passed - TypeScript: `pnpm build`; `table`, `query` tests: 606 passed; `biome check` clean; `docs/src/js` regenerated with `pnpm run docs` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
1772f4ef83
commit
92825f6f13
+14
-14
@@ -333,7 +333,7 @@ pub struct PyQueryRequest {
|
||||
pub select_source_columns: Option<HashMap<String, String>>,
|
||||
pub fast_search: Option<bool>,
|
||||
pub with_row_id: Option<bool>,
|
||||
pub use_memwal: Option<bool>,
|
||||
pub use_rte: Option<bool>,
|
||||
pub column: Option<String>,
|
||||
pub query_vector: Option<PyQueryVectors>,
|
||||
pub nprobes: Option<usize>,
|
||||
@@ -366,7 +366,7 @@ impl From<AnyQuery> for PyQueryRequest {
|
||||
select: PySelect(query_request.select),
|
||||
fast_search: Some(query_request.fast_search),
|
||||
with_row_id: Some(query_request.with_row_id),
|
||||
use_memwal: query_request.use_memwal,
|
||||
use_rte: query_request.use_rte,
|
||||
column: None,
|
||||
query_vector: None,
|
||||
nprobes: None,
|
||||
@@ -395,7 +395,7 @@ impl From<AnyQuery> for PyQueryRequest {
|
||||
select: PySelect(vector_query.base.select),
|
||||
fast_search: Some(vector_query.base.fast_search),
|
||||
with_row_id: Some(vector_query.base.with_row_id),
|
||||
use_memwal: vector_query.base.use_memwal,
|
||||
use_rte: vector_query.base.use_rte,
|
||||
column: vector_query.column,
|
||||
query_vector: Some(PyQueryVectors(vector_query.query_vector)),
|
||||
nprobes: vector_query.nprobes,
|
||||
@@ -540,8 +540,8 @@ impl Query {
|
||||
self.inner = self.inner.clone().fast_search();
|
||||
}
|
||||
|
||||
pub fn use_memwal(&mut self, enable: bool) {
|
||||
self.inner = self.inner.clone().use_memwal(enable);
|
||||
pub fn use_rte(&mut self, enable: bool) {
|
||||
self.inner = self.inner.clone().use_rte(enable);
|
||||
}
|
||||
|
||||
pub fn with_row_id(&mut self) {
|
||||
@@ -706,8 +706,8 @@ impl TakeQuery {
|
||||
self.inner = self.inner.clone().with_row_id();
|
||||
}
|
||||
|
||||
pub fn use_memwal(&mut self, enable: bool) {
|
||||
self.inner = self.inner.clone().use_memwal(enable);
|
||||
pub fn use_rte(&mut self, enable: bool) {
|
||||
self.inner = self.inner.clone().use_rte(enable);
|
||||
}
|
||||
|
||||
#[pyo3(signature = ())]
|
||||
@@ -819,8 +819,8 @@ impl FTSQuery {
|
||||
self.inner = self.inner.clone().fast_search();
|
||||
}
|
||||
|
||||
pub fn use_memwal(&mut self, enable: bool) {
|
||||
self.inner = self.inner.clone().use_memwal(enable);
|
||||
pub fn use_rte(&mut self, enable: bool) {
|
||||
self.inner = self.inner.clone().use_rte(enable);
|
||||
}
|
||||
|
||||
pub fn with_row_id(&mut self) {
|
||||
@@ -970,8 +970,8 @@ impl VectorQuery {
|
||||
self.inner = self.inner.clone().fast_search();
|
||||
}
|
||||
|
||||
pub fn use_memwal(&mut self, enable: bool) {
|
||||
self.inner = self.inner.clone().use_memwal(enable);
|
||||
pub fn use_rte(&mut self, enable: bool) {
|
||||
self.inner = self.inner.clone().use_rte(enable);
|
||||
}
|
||||
|
||||
pub fn with_row_id(&mut self) {
|
||||
@@ -1178,9 +1178,9 @@ impl HybridQuery {
|
||||
self.inner_fts.postfilter();
|
||||
}
|
||||
|
||||
pub fn use_memwal(&mut self, enable: bool) {
|
||||
self.inner_vec.use_memwal(enable);
|
||||
self.inner_fts.use_memwal(enable);
|
||||
pub fn use_rte(&mut self, enable: bool) {
|
||||
self.inner_vec.use_rte(enable);
|
||||
self.inner_fts.use_rte(enable);
|
||||
}
|
||||
|
||||
pub fn add_query_vector(&mut self, vector: Bound<'_, PyAny>) -> PyResult<()> {
|
||||
|
||||
Reference in New Issue
Block a user