mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-22 11:42:27 +00:00
docs: add realtime engine (RTE) getting-started guide (#4548)
Adds a user-facing realtime engine (RTE) page at `docs/web/tables/realtime-engine.mdx`, listed under **Management** after Branches. It is adapted from the internal WAL field guide (lancedb/sophon#6213), with the deployment and operations material left out. The page covers: - what RTE is and how it works (memtable → SSTables → compaction; reads merge the base table with the fresh tier) - getting started: `set_unenforced_primary_key`, indexes, `enable_rte` / `ALTER TABLE ... SET RTE`, `merge_insert` upserts and queries, partial updates, deletes, and `checkpoint_rte` / `disable_rte` - performance: write throughput and shard scaling, and read latency with and without RTE - a note on operations not yet supported (time travel, branching, `count_rows`, row IDs, `TABLESAMPLE`) Also aligns related docs with current RTE naming and behavior: - `sql/ddl/table.mdx`: `SET MEMWAL` / `UNSET MEMWAL` → `SET RTE` / `UNSET RTE` - `maintenance/background-jobs.mdx`: job names are `memwal_compaction` and `memwal_catchup_<type>` - `feature-engineering/materialized-views.mdx`: MemWAL → realtime engine - Python `AsyncTable.enable_rte` / `RTEWriteSpec.with_maintained_indexes` docstrings: by default, indexes created after enabling RTE are maintained too The examples are written directly in the page rather than generated from `docs/web-tests`, because RTE needs an Enterprise server to run. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: XY Zhan <zhaner08@hotmail.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
XY Zhan
parent
9e54c8cf27
commit
951ae51ef6
@@ -850,9 +850,9 @@ class RTEWriteSpec:
|
||||
@staticmethod
|
||||
def unsharded() -> "RTEWriteSpec": ...
|
||||
def with_maintained_indexes(self, indexes: Optional[List[str]]) -> "RTEWriteSpec":
|
||||
"""Set which indexes the RTE keeps up to date. None resolves every
|
||||
index on the table at install, failing if one cannot be maintained;
|
||||
a list is verbatim, empty means none."""
|
||||
"""Set which indexes the RTE keeps up to date. None is every index on
|
||||
the table, now and later, skipping any it cannot maintain; a list is
|
||||
verbatim, empty means none."""
|
||||
...
|
||||
def with_writer_config_defaults(self, defaults: Dict[str, str]) -> "RTEWriteSpec":
|
||||
"""Return a copy of this spec recording the given default
|
||||
|
||||
@@ -5564,12 +5564,12 @@ class AsyncTable:
|
||||
via [`set_unenforced_primary_key`]; bucket sharding additionally
|
||||
requires it to be the single column being bucketed.
|
||||
|
||||
By default the RTE maintains every index on the table, resolved
|
||||
here — a snapshot, so an index created afterwards needs the spec unset
|
||||
and set again. This fails if one cannot be maintained; name the set
|
||||
with ``with_maintained_indexes`` to install anyway. That pins an exact
|
||||
set (a still-building index is rejected, not omitted); ``[]`` maintains
|
||||
none.
|
||||
By default the RTE maintains every index on the table, re-read as the
|
||||
table changes, so an index created later is maintained too; one of a
|
||||
kind the RTE cannot mirror is skipped. Name the set with
|
||||
``with_maintained_indexes`` to pin it instead: each name must already
|
||||
exist and be maintainable (a still-building index is rejected, not
|
||||
omitted); ``[]`` maintains none.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
|
||||
Reference in New Issue
Block a user