fix: align table listing pagination across backends (#4412)

Fixes #4379

Remote `table_names` discarded the continuation token after the first
root-listing request, silently returning the server's default page of 10
names. Remote `list_tables` also inherited that server default, while
local listings had no SDK page limit. Python's synchronous and namespace
wrappers still imposed a default limit of 10 names. The namespace
adapter also discarded continuation tokens, so an unlimited REST
namespace listing could stop at the server's page cap.

Follow server-provided root listing tokens until the requested name
limit or the end of the listing, preserving `start_after`, cache
population, and guards against empty or repeated tokens. The namespace
adapter collects every server page, then sorts names and applies the
name cursor and total limit: namespace tokens are opaque, and page order
need not be name order. Empty tokens end the listing; token cycles
return an error. Python sync and namespace `table_names` wrappers now
default to `None`, matching the asynchronous and TypeScript APIs.
Namespace overrides accept the base connection's optional limit,
avoiding import-time signature errors on Python 3.10 and 3.11. Set the
`list_tables` default to 100 at the shared connection boundary,
including namespaces, and return empty results for a zero limit without
sending an invalid server request. Update the SDK documentation to
describe these defaults.

`list_tables()` is now a paginated call in both modes: callers with more
than 100 tables must follow the returned token or supply an explicit
larger limit. `table_names()` without a limit returns every name.
Namespace table-name calls enumerate the namespace before applying name
cursors and limits; use `list_tables()` for paginated access.

Validation:

- Regression cases reproduce the original root-listing truncation and
cover 130 tables, opaque tokens, short pages, explicit limits,
`start_after`, and zero limits.
- Relevant Rust remote database, connection, and local listing test
groups: 175 passed. Namespace adapter tests: 23 passed, including opaque
tokens, unordered and empty pages, name cursors, limits, zero requests,
and token cycles.
- Rebuilt the Python extension; namespace and local/mocked remote
listing tests using `uv.lock`: 46 passed on Python 3.10 and 46 passed on
Python 3.13.
- REST namespace regressions reproduced the truncation before the fix
and now return all 12 names from a server capped at 10 per response, for
sync/async connections at root and child paths. They also cover
unordered pages, name cursors, explicit limits, and zero requests. The
exact Python 3.10 import error also reproduced before the
optional-signature fix.
- TypeScript connection test file: 38 passed; `pnpm build`, `pnpm lint`,
and `pnpm run docs` passed.
- `cargo check --quiet --features remote --tests --examples` and `cargo
clippy --quiet --features remote --tests --examples` passed.
- Rust formatting, Python Ruff formatting/lint, and diff whitespace
checks passed.

<!-- lance-gatekeeper-fix:v1 agent=43f92f1f05ac9ca74229af590945cb88
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>

python/python/tests/test_remote_db.py

@@ -13,6 +13,7 @@ import threading
 import time
 from unittest.mock import MagicMock, patch
 import uuid
+from urllib.parse import parse_qs, urlparse
 from packaging.version import Version
 
 import lancedb
@@ -105,6 +106,67 @@ async def mock_lancedb_connection_async(handler, **client_config):
             handle.join()
 
 
+def paginated_table_listing_handler(request):
+    url = urlparse(request.path)
+    assert url.path in {
+        "/v1/table/",
+        "/v1/namespace/$/table/list",
+        "/v1/namespace/n/table/list",
+    }
+    query = parse_qs(url.query)
+    limit = int(query.get("limit", [10])[0])
+    assert limit > 0
+    token = query.get("page_token", ["page-0"])[0]
+    start = int(token.removeprefix("page-"))
+    end = min(start + limit, 130)
+    body = {"tables": [f"t{i:03}" for i in range(start, end)]}
+    if end < 130:
+        body["page_token"] = f"page-{end}"
+    request.send_response(200)
+    request.send_header("Content-Type", "application/json")
+    request.end_headers()
+    request.wfile.write(json.dumps(body).encode())
+
+
+@pytest.mark.parametrize("namespace", [[], ["n"]])
+def test_remote_table_listing_defaults_and_zero_limit(namespace):
+    names = [f"t{i:03}" for i in range(130)]
+    with mock_lancedb_connection(paginated_table_listing_handler) as db:
+        assert db.table_names(namespace_path=namespace) == names
+        assert db.table_names(limit=None, namespace_path=namespace) == names
+        assert db.table_names(limit=0, namespace_path=namespace) == []
+        first = db.list_tables(namespace_path=namespace)
+        assert first.tables == names[:100]
+        assert first.page_token == "page-100"
+        second = db.list_tables(namespace_path=namespace, page_token=first.page_token)
+        assert second.tables == names[100:]
+        assert second.page_token is None
+        assert db.list_tables(namespace_path=namespace, limit=200).tables == names
+        empty = db.list_tables(namespace_path=namespace, limit=0)
+        assert empty.tables == []
+        assert empty.page_token is None
+
+
+@pytest.mark.asyncio
+@pytest.mark.parametrize("namespace", [[], ["n"]])
+async def test_async_remote_table_listing_defaults_and_zero_limit(namespace):
+    names = [f"t{i:03}" for i in range(130)]
+    async with mock_lancedb_connection_async(paginated_table_listing_handler) as db:
+        assert await db.table_names(namespace_path=namespace) == names
+        assert await db.table_names(limit=0, namespace_path=namespace) == []
+        first = await db.list_tables(namespace_path=namespace)
+        assert first.tables == names[:100]
+        assert first.page_token == "page-100"
+        second = await db.list_tables(
+            namespace_path=namespace, page_token=first.page_token
+        )
+        assert second.tables == names[100:]
+        assert second.page_token is None
+        empty = await db.list_tables(namespace_path=namespace, limit=0)
+        assert empty.tables == []
+        assert empty.page_token is None
+
+
 @pytest.mark.asyncio
 async def test_async_remote_db():
     def handler(request):