Dan RammerandClaude Opus 5.5
92825f6f13
## What Renames the user-facing MemWAL surface to **RTE** (realtime engine) across Rust, Python, TypeScript, and Java, including the REST wire format. This follows the LSM → MemWAL rename in #4075. | Before | After | |---|---| | `MemWalWriteSpec` / `MemWalStats` | `RTEWriteSpec` / `RTEStats` | | `enable_memwal` / `disable_memwal` / `get_memwal_spec` | `enable_rte` / `disable_rte` / `get_rte_spec` | | `flush_memwal` / `compact_memwal` / `checkpoint_memwal` | `flush_rte` / `compact_rte` / `checkpoint_rte` | | `get_memwal_stats` / `close_memwal_writers` | `get_rte_stats` / `close_rte_writers` | | `use_memwal` (query, merge insert) | `use_rte` | | TS/Java `enableMemWal`, `useMemWal`, ... | `enableRTE`, `useRTE`, ... (napi `js_name` set explicitly) | | Java `LanceDbTableMemWal` | `LanceDbTableRTE` | | `table/memwal_stats.rs` | `table/rte_stats.rs` | Docs and comments now say "realtime engine (RTE)". Text about Lance's underlying mechanism keeps "MemWAL" ("Lance's MemWAL", "MemWAL index"). **Left alone:** Lance-owned names (`MemWalIndexDetails`, `DatasetMemWalExt`, `InitializeMemWal*`, `mem_wal` paths), internal `Lsm*` names and modules, and the `does not support with_row_id` refusal text that the hybrid fallback matches on. ## Compatibility Breaking, for an experimental API. The wire format changes, so this must roll out together with the matching sophon change (lancedb/sophon#8870): - Routes: `/enable_memwal/` → `/enable_rte/`, `/disable_memwal/` → `/disable_rte/`, `/get_memwal_spec/` → `/get_rte_spec/`, `/get_memwal_stats/` → `/get_rte_stats/`, `/flush_memwal/` → `/flush_rte/`, `/compact_memwal/` → `/compact_rte/` - Response keys: `memwal_spec` → `rte_spec`, `memwal_stats` → `rte_stats` - Request field: `use_memwal` → `use_rte` on query and merge-insert bodies The SQL keyword becomes `ALTER TABLE ... SET RTE` server-side. The guides under `docs/web` will be updated separately. ## Validation - Rust: `cargo clippy --features remote --tests --examples` clean; `cargo test --features remote -p lancedb --lib`: 1533 passed - Java: `mvn test`: 37 passed (Spotless applied with JDK 11) - Python: `ruff format` / `ruff check` clean; targeted tests (`test_rte_spec`, `test_merge_insert_lsm`, `test_remote_db`, `test_wal_hybrid`, `test_query`, `test_permutation`, `test_table`): 708 passed - TypeScript: `pnpm build`; `table`, `query` tests: 606 passed; `biome check` clean; `docs/src/js` regenerated with `pnpm run docs` 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
4.5 KiB
@lancedb/lancedb • Docs
@lancedb/lancedb / MergeInsertBuilder
Class: MergeInsertBuilder
A builder used to create and run a merge insert operation
Constructors
new MergeInsertBuilder()
new MergeInsertBuilder(native, schema): MergeInsertBuilder
Construct a MergeInsertBuilder. Internal use only.
Parameters
-
native:
NativeMergeInsertBuilder -
schema:
Schema<any> |Promise<Schema<any>>
Returns
Methods
execute()
execute(data, execOptions?): Promise<MergeResult>
Executes the merge insert operation
Parameters
-
data:
Data -
execOptions?:
Partial<WriteExecutionOptions>
Returns
Promise<MergeResult>
the merge result
useIndex()
useIndex(useIndex): MergeInsertBuilder
Controls whether to use indexes for the merge operation.
When set to true (the default), the operation will use an index if available
on the join key for improved performance. When set to false, it forces a full
table scan even if an index exists. This can be useful for benchmarking or when
the query optimizer chooses a suboptimal path.
Parameters
- useIndex:
booleanWhether to use indices for the merge operation. Defaults totrue.
Returns
useRTE()
useRTE(enable): MergeInsertBuilder
Control RTE routing for this merge.
By default (unset), a mergeInsert on a table with an RTE write spec is
routed through Lance's MemWAL shard writer, and a table without one uses the
standard path.
Parameters
- enable:
booleantrueforces RTE routing and errors if the table has no RTE write spec.falseforces the standard write path even when a spec is set.
Returns
validateSingleShard()
validateSingleShard(validateSingleShard): MergeInsertBuilder
Controls how an LSM merge checks that its input targets a single shard.
When a table has an RTE write spec, every row in a mergeInsert call must
route to the same shard. When true (the default), every row is inspected
to verify this. When false, only the first row is inspected and the
shard it routes to is used for the whole input — a faster path for callers
that have already pre-sharded their input. Has no effect on tables without
an RTE write spec.
Parameters
- validateSingleShard:
booleanWhether to check every row routes to one shard. Defaults totrue.
Returns
whenMatchedUpdateAll()
whenMatchedUpdateAll(options?): MergeInsertBuilder
Rows that exist in both the source table (new data) and the target table (old data) will be updated, replacing the old row with the corresponding matching row.
If there are multiple matches then the behavior is undefined. Currently this causes multiple copies of the row to be created but that behavior is subject to change.
An optional condition may be specified. If it is, then only matched rows that satisfy the condition will be updated. Any rows that do not satisfy the condition will be left as they are. Failing to satisfy the condition does not cause a "matched row" to become a "not matched" row.
The condition should be an SQL string. Use the prefix target. to refer to rows in the target table (old data) and the prefix source. to refer to rows in the source table (new data).
For example, "target.last_update < source.last_update"
Parameters
-
options?
-
options.where?:
string
Returns
whenNotMatchedBySourceDelete()
whenNotMatchedBySourceDelete(options?): MergeInsertBuilder
Rows that exist only in the target table (old data) will be deleted. An optional condition can be provided to limit what data is deleted.
Parameters
-
options?
-
options.where?:
stringAn optional condition to limit what data is deleted
Returns
whenNotMatchedInsertAll()
whenNotMatchedInsertAll(): MergeInsertBuilder
Rows that exist only in the source table (new data) should be inserted into the target table.