mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-23 20:22:37 +00:00
## What Renames the user-facing MemWAL surface to **RTE** (realtime engine) across Rust, Python, TypeScript, and Java, including the REST wire format. This follows the LSM → MemWAL rename in #4075. | Before | After | |---|---| | `MemWalWriteSpec` / `MemWalStats` | `RTEWriteSpec` / `RTEStats` | | `enable_memwal` / `disable_memwal` / `get_memwal_spec` | `enable_rte` / `disable_rte` / `get_rte_spec` | | `flush_memwal` / `compact_memwal` / `checkpoint_memwal` | `flush_rte` / `compact_rte` / `checkpoint_rte` | | `get_memwal_stats` / `close_memwal_writers` | `get_rte_stats` / `close_rte_writers` | | `use_memwal` (query, merge insert) | `use_rte` | | TS/Java `enableMemWal`, `useMemWal`, ... | `enableRTE`, `useRTE`, ... (napi `js_name` set explicitly) | | Java `LanceDbTableMemWal` | `LanceDbTableRTE` | | `table/memwal_stats.rs` | `table/rte_stats.rs` | Docs and comments now say "realtime engine (RTE)". Text about Lance's underlying mechanism keeps "MemWAL" ("Lance's MemWAL", "MemWAL index"). **Left alone:** Lance-owned names (`MemWalIndexDetails`, `DatasetMemWalExt`, `InitializeMemWal*`, `mem_wal` paths), internal `Lsm*` names and modules, and the `does not support with_row_id` refusal text that the hybrid fallback matches on. ## Compatibility Breaking, for an experimental API. The wire format changes, so this must roll out together with the matching sophon change (lancedb/sophon#8870): - Routes: `/enable_memwal/` → `/enable_rte/`, `/disable_memwal/` → `/disable_rte/`, `/get_memwal_spec/` → `/get_rte_spec/`, `/get_memwal_stats/` → `/get_rte_stats/`, `/flush_memwal/` → `/flush_rte/`, `/compact_memwal/` → `/compact_rte/` - Response keys: `memwal_spec` → `rte_spec`, `memwal_stats` → `rte_stats` - Request field: `use_memwal` → `use_rte` on query and merge-insert bodies The SQL keyword becomes `ALTER TABLE ... SET RTE` server-side. The guides under `docs/web` will be updated separately. ## Validation - Rust: `cargo clippy --features remote --tests --examples` clean; `cargo test --features remote -p lancedb --lib`: 1533 passed - Java: `mvn test`: 37 passed (Spotless applied with JDK 11) - Python: `ruff format` / `ruff check` clean; targeted tests (`test_rte_spec`, `test_merge_insert_lsm`, `test_remote_db`, `test_wal_hybrid`, `test_query`, `test_permutation`, `test_table`): 708 passed - TypeScript: `pnpm build`; `table`, `query` tests: 606 passed; `biome check` clean; `docs/src/js` regenerated with `pnpm run docs` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
90 lines
2.5 KiB
Rust
90 lines
2.5 KiB
Rust
// SPDX-License-Identifier: Apache-2.0
|
|
// SPDX-FileCopyrightText: Copyright The LanceDB Authors
|
|
|
|
use std::time::Duration;
|
|
|
|
use lancedb::{ipc::ipc_file_to_batches, table::merge::MergeInsertBuilder};
|
|
use napi::bindgen_prelude::*;
|
|
use napi_derive::napi;
|
|
|
|
use crate::{error::convert_error, table::MergeResult};
|
|
|
|
#[napi]
|
|
#[derive(Clone)]
|
|
/// A builder used to create and run a merge insert operation
|
|
pub struct NativeMergeInsertBuilder {
|
|
pub(crate) inner: MergeInsertBuilder,
|
|
}
|
|
|
|
#[napi]
|
|
impl NativeMergeInsertBuilder {
|
|
#[napi]
|
|
pub fn when_matched_update_all(&self, condition: Option<String>) -> Self {
|
|
let mut this = self.clone();
|
|
this.inner.when_matched_update_all(condition);
|
|
this
|
|
}
|
|
|
|
#[napi]
|
|
pub fn when_not_matched_insert_all(&self) -> Self {
|
|
let mut this = self.clone();
|
|
this.inner.when_not_matched_insert_all();
|
|
this
|
|
}
|
|
#[napi]
|
|
pub fn when_not_matched_by_source_delete(&self, filter: Option<String>) -> Self {
|
|
let mut this = self.clone();
|
|
this.inner.when_not_matched_by_source_delete(filter);
|
|
this
|
|
}
|
|
|
|
#[napi]
|
|
pub fn set_timeout(&mut self, timeout: u32) {
|
|
self.inner.timeout(Duration::from_millis(timeout as u64));
|
|
}
|
|
|
|
#[napi]
|
|
pub fn use_index(&self, use_index: bool) -> Self {
|
|
let mut this = self.clone();
|
|
this.inner.use_index(use_index);
|
|
this
|
|
}
|
|
|
|
#[napi(js_name = "useRTE")]
|
|
pub fn use_rte(&self, enable: bool) -> Self {
|
|
let mut this = self.clone();
|
|
this.inner.use_rte(enable);
|
|
this
|
|
}
|
|
|
|
#[napi]
|
|
pub fn validate_single_shard(&self, validate_single_shard: bool) -> Self {
|
|
let mut this = self.clone();
|
|
this.inner.validate_single_shard(validate_single_shard);
|
|
this
|
|
}
|
|
|
|
#[napi(catch_unwind)]
|
|
pub async fn execute(&self, buf: Buffer) -> napi::Result<MergeResult> {
|
|
let data = ipc_file_to_batches(buf.to_vec()).map_err(|e| {
|
|
napi::Error::from_reason(format!("Failed to read IPC file: {}", convert_error(&e)))
|
|
})?;
|
|
|
|
let this = self.clone();
|
|
|
|
let res = this.inner.execute(data).await.map_err(|e| {
|
|
napi::Error::from_reason(format!(
|
|
"Failed to execute merge insert: {}",
|
|
convert_error(&e)
|
|
))
|
|
})?;
|
|
Ok(res.into())
|
|
}
|
|
}
|
|
|
|
impl From<MergeInsertBuilder> for NativeMergeInsertBuilder {
|
|
fn from(inner: MergeInsertBuilder) -> Self {
|
|
Self { inner }
|
|
}
|
|
}
|