feat(sql): follow adaptive streaming and polling responses (#4664)

SQL execution now starts with one Flight call. The server can stream
rows immediately or return a polling continuation, which the SDK follows
through the existing query handle and reader APIs. The same behavior
applies with and without parameters.

Both initial transports share direct-stream lifecycle handling. Handoffs
are explicitly negotiated; existing servers can continue returning
ordinary streams. Add coverage for streaming, deferred results,
cancellation, malformed decisions, and legacy responses.

```mermaid
flowchart TD
    A[execute_query or execute_query_async] --> B{Parameters supplied?}
    B -->|No| C[DoGet: SQL command ticket]
    B -->|Yes| D[DoExchange: SQL and Arrow parameter row]
    C --> E[Server chooses execution path]
    D --> E
    E -->|Stream| F[Initial response: streaming decision and Arrow batches]
    E -->|Defer| G[Initial response: PollInfo continuation]
    G --> H[SDK follows PollFlightInfo]
    H -->|Still running| H
    H -->|Result endpoints available| I[SDK fetches tickets with DoGet]
    F --> J[Query reader yields Arrow batches]
    I --> J
```

`execute_query` opens the reader; `execute_query_async` returns its
query handle. The caller does not select the transport or execution
path.
This commit is contained in:
Jack Ye
2026-10-25 15:47:29 +08:00
committed by GitHub
parent 6b6778e684
commit 24c2669a50
8 changed files with 414 additions and 133 deletions
+11 -2
View File
@@ -221,10 +221,19 @@ The connected database and `default_namespace_path=["public"]` are used for
unqualified tables. Fully qualified references can still query other databases
and namespaces available to the same deployment. `execute_query` returns a
reader as soon as its initial result stream is available. `execute_query_async`
returns a query handle immediately; use it to inspect progress, open a reader,
returns a query handle once submission is accepted; use it to inspect progress, open a reader,
or cancel the query. The SQL client is initialized by the first query and
retained for the lifetime of the remote connection. Query ids are random,
connection-scoped references rather than encoded SQL or durable resume tokens:
connection-scoped references rather than encoded SQL or durable resume tokens.
Both APIs start with one Flight call. The server can stream results on that
call or return a polling continuation that the SDK follows automatically.
This behavior is the same with and without parameters. Open the reader directly;
a directly streamed query needs its reader to make progress. Dropping an unread
handle or its reader stops direct streaming. Deferred execution continues until
explicitly cancelled.
For example:
```python
import lancedb
+2
View File
@@ -82,6 +82,8 @@ In application code, load credentials from environment variables or a secrets
manager. The [connection guide](/basics/connect#enterprise) shows that setup.
`execute_query()` returns a blocking Arrow reader when results start arriving.
The server can stream rows on the first call or return result tickets that the
SDK follows automatically, with the same behavior for parameterized queries.
The example uses `read_all()` to collect the small result in memory. For larger
results, consume the reader in batches. To submit a query and check its status
separately, use `execute_query_async()`. See [Queries](/basics/query) for query