feat(table): checkpoint_lsm, flush_lsm, compact_lsm, get_lsm_stats

Converge a table's LSM write path into its base table, and inspect it.

`checkpoint_lsm` is `flush` then `compact`, repeated until the fresh tier
is empty — and the loop runs **client-side**. Putting it on the server
would mean a background task, which means a single-flight intent, an
intent that leaks on panic, a bounded-iteration policy, an "is it done"
observable, and a story for every way a client can vanish mid-operation.
None of that exists in this shape: each request does a bounded unit of
work and reports what is left, so completion is *carried in the
responses* rather than inferred from a shared counter that cannot
distinguish "converged" from "hasn't started yet".

Best-effort by construction. Nothing is frozen, so `converged` means L0
was empty as of the last pass. It is idempotent, abandonable at any point
with zero consequence, and safe to run on a cadence — an already-
converged table costs one round trip and zero compaction passes, because
`flush` reports `generations_remaining` and the loop is never entered.

The failure taxonomy is the load-bearing part. Five distinct conditions
used to arrive at a client as one 503, so `Error::LsmRoute` carries a
classification read from the response body's namespace error code *at
the point of receipt* — before any generic helper folds the body into a
string and keeps only the status. Contention (429) retries; a draining
node (503 / `InvalidTableState`) stops, because the drain gate is a
one-way latch and retrying it spins until the deadline to report a
failure that was knowable on the first response. Transport retry is
disabled on these routes for the same reason: it treats every 503 alike.

`get_lsm_stats` returns `Option<LsmStats>`, matching `get_lsm_write_spec`
— `None` only when the table has no LSM write path, since a struct of
zeros would read as measurements.

Python bindings mirror all four, preserving the per-bucket detail rather
than flattening to a table-level summary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

python/python/lancedb/_lancedb.pyi

@@ -284,6 +284,15 @@ class Table:
     async def set_lsm_write_spec(self, spec: LsmWriteSpec) -> None: ...
     async def unset_lsm_write_spec(self) -> None: ...
     async def get_lsm_write_spec(self) -> Optional[LsmWriteSpec]: ...
+    async def checkpoint_lsm(
+        self,
+        deadline_secs: Optional[float],
+        max_generations_per_bucket: Optional[int],
+        include_stats: bool,
+    ) -> dict: ...
+    async def flush_lsm(self) -> dict: ...
+    async def compact_lsm(self, max_generations_per_bucket: Optional[int]) -> dict: ...
+    async def get_lsm_stats(self, include_generation_rows: bool) -> Optional[dict]: ...
     async def close_lsm_writers(self) -> None: ...
     @property
     def tags(self) -> Tags: ...

python/python/lancedb/table.py

@@ -3872,6 +3872,44 @@ class LanceTable(Table):
         [`AsyncTable.get_lsm_write_spec`][lancedb.AsyncTable.get_lsm_write_spec]."""
         return LOOP.run(self._table.get_lsm_write_spec())
 
+    def checkpoint_lsm(
+        self,
+        *,
+        deadline_secs: Optional[float] = 300.0,
+        max_generations_per_bucket: Optional[int] = None,
+        include_stats: bool = False,
+    ) -> dict:
+        """Synchronous version of
+        [`AsyncTable.checkpoint_lsm`][lancedb.AsyncTable.checkpoint_lsm]."""
+        return LOOP.run(
+            self._table.checkpoint_lsm(
+                deadline_secs=deadline_secs,
+                max_generations_per_bucket=max_generations_per_bucket,
+                include_stats=include_stats,
+            )
+        )
+
+    def flush_lsm(self) -> dict:
+        """Synchronous version of
+        [`AsyncTable.flush_lsm`][lancedb.AsyncTable.flush_lsm]."""
+        return LOOP.run(self._table.flush_lsm())
+
+    def compact_lsm(self, *, max_generations_per_bucket: Optional[int] = None) -> dict:
+        """Synchronous version of
+        [`AsyncTable.compact_lsm`][lancedb.AsyncTable.compact_lsm]."""
+        return LOOP.run(
+            self._table.compact_lsm(
+                max_generations_per_bucket=max_generations_per_bucket
+            )
+        )
+
+    def get_lsm_stats(self, *, include_generation_rows: bool = False) -> Optional[dict]:
+        """Synchronous version of
+        [`AsyncTable.get_lsm_stats`][lancedb.AsyncTable.get_lsm_stats]."""
+        return LOOP.run(
+            self._table.get_lsm_stats(include_generation_rows=include_generation_rows)
+        )
+
     def close_lsm_writers(self) -> None:
         """Close cached MemWAL shard writers. See
         [`AsyncTable.close_lsm_writers`][lancedb.AsyncTable.close_lsm_writers]."""
@@ -4582,6 +4620,83 @@ class AsyncTable:
         """
         return await self._inner.get_lsm_write_spec()
 
+    async def checkpoint_lsm(
+        self,
+        *,
+        deadline_secs: Optional[float] = 300.0,
+        max_generations_per_bucket: Optional[int] = None,
+        include_stats: bool = False,
+    ) -> dict:
+        """Converge this table's LSM write path into its base table.
+
+        One flush (sealing every memtable into L0), then bounded compaction
+        passes until L0 is empty. The loop runs client-side, so each request
+        does a bounded unit of work and reports what is left.
+
+        Best-effort: nothing is frozen, so with writes flowing there may be
+        new rows by the time it returns — ``converged`` means L0 was empty as
+        of the last pass. Idempotent and safe on a cadence; an
+        already-converged table costs one round trip and zero compaction
+        passes.
+
+        Parameters
+        ----------
+        deadline_secs
+            Stop and report ``converged=False`` after this much wall clock.
+            ``None`` runs until the table converges — safe only when you know
+            it is not under write load.
+        max_generations_per_bucket
+            Bound on generations merged **per bucket** per request. An
+            N-bucket table admits ``N * max_generations_per_bucket`` merges
+            per call.
+        include_stats
+            Attach a ``get_lsm_stats`` block to the report.
+        """
+        return await self._inner.checkpoint_lsm(
+            deadline_secs, max_generations_per_bucket, include_stats
+        )
+
+    async def flush_lsm(self) -> dict:
+        """Seal every bucket's active memtable into L0.
+
+        Does not touch the base table — moving L0 into base is
+        `compact_lsm`. On a node that has not claimed this table, this claims
+        it and replays its WAL log first.
+        """
+        return await self._inner.flush_lsm()
+
+    async def compact_lsm(
+        self, *, max_generations_per_bucket: Optional[int] = None
+    ) -> dict:
+        """Run one bounded L0 to base compaction pass per bucket.
+
+        One pass, not convergence: that is what bounds each request's cost
+        and gives a caller driving its own cadence a progress signal per
+        round trip. Use `checkpoint_lsm` to loop to convergence.
+        """
+        return await self._inner.compact_lsm(max_generations_per_bucket)
+
+    async def get_lsm_stats(
+        self, *, include_generation_rows: bool = False
+    ) -> Optional[dict]:
+        """Read live LSM state: per-bucket detail plus a table-level aggregate.
+
+        Answers "how far behind is my fresh tier", "which bucket is hot", and
+        "why is my fresh-tier vector search brute-force". Mutates no table
+        state, though on a node that has not claimed this table it claims it,
+        exactly as a read would.
+
+        Returns ``None`` — and only — when the LSM write path is not enabled
+        for the table.
+
+        Parameters
+        ----------
+        include_generation_rows
+            Report row counts per L0 generation. Off by default: each costs
+            one Lance manifest read.
+        """
+        return await self._inner.get_lsm_stats(include_generation_rows)
+
     async def close_lsm_writers(self) -> None:
         """Drain and close any cached MemWAL shard writers for this table.