rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-10-22 11:42:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: add realtime engine (RTE) getting-started guide (#4548)

Browse Source 
Adds a user-facing realtime engine (RTE) page at
`docs/web/tables/realtime-engine.mdx`, listed under **Management** after
Branches. It is adapted from the internal WAL field guide
(lancedb/sophon#6213), with the deployment and operations material left
out.

The page covers:
- what RTE is and how it works (memtable → SSTables → compaction; reads
merge the base table with the fresh tier)
- getting started: `set_unenforced_primary_key`, indexes, `enable_rte` /
`ALTER TABLE ... SET RTE`, `merge_insert` upserts and queries, partial
updates, deletes, and `checkpoint_rte` / `disable_rte`
- performance: write throughput and shard scaling, and read latency with
and without RTE
- a note on operations not yet supported (time travel, branching,
`count_rows`, row IDs, `TABLESAMPLE`)

Also aligns related docs with current RTE naming and behavior:
- `sql/ddl/table.mdx`: `SET MEMWAL` / `UNSET MEMWAL` → `SET RTE` /
`UNSET RTE`
- `maintenance/background-jobs.mdx`: job names are `memwal_compaction`
and `memwal_catchup_<type>`
- `feature-engineering/materialized-views.mdx`: MemWAL → realtime engine
- Python `AsyncTable.enable_rte` /
`RTEWriteSpec.with_maintained_indexes` docstrings: by default, indexes
created after enabling RTE are maintained too

The examples are written directly in the page rather than generated from
`docs/web-tests`, because RTE needs an Enterprise server to run.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: XY Zhan <zhaner08@hotmail.com>
This commit is contained in:
Dan Rammer
2026-10-20 21:40:20 -05:00
committed by GitHub
co-authored by Claude Opus 5.5 XY Zhan
parent 9e54c8cf27
commit 951ae51ef6
7 changed files with 251 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/web/sql/ddl/table.mdx
+8 -6
View File
@@ -102,18 +102,20 @@ duplicates impossible. It tells the planner what to assume. The column must be
declared `NOT NULL`.
```sql
ALTER TABLE t SET MEMWAL
ALTER TABLE t SET MEMWAL WITH (sharding = 'bucket', column = 'id', buckets = 4)
ALTER TABLE t SET MEMWAL WITH (maintained_indexes = '')
ALTER TABLE t UNSET MEMWAL
ALTER TABLE t SET RTE
ALTER TABLE t SET RTE WITH (sharding = 'bucket', column = 'id', buckets = 4)
ALTER TABLE t SET RTE WITH (maintained_indexes = '')
ALTER TABLE t UNSET RTE
```
A write-ahead log makes small frequent writes cheap by batching them out of the
The [realtime engine](/tables/realtime-engine) puts a write-ahead log in front
of the table, making small frequent writes cheap by batching them out of the
commit path. `sharding` and `buckets` spread the log so concurrent writers do
not contend on one place; `bucket` sharding names its `column`, which must be
the table's unenforced primary key, and inserts into the table then need a
scalar index on that key. `maintained_indexes` chooses which indexes are kept
current against WAL rows. A table with a computed column cannot take a
current against WAL rows; omitted, every supported index is kept current,
including ones created later. A table with a computed column cannot take a
write-ahead log. See
[Background jobs](/maintenance/background-jobs#wal) for how the log gets
folded back into the table.