mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-26 13:36:13 +00:00
feat(sql): follow adaptive streaming and polling responses (#4664)
SQL execution now starts with one Flight call. The server can stream
rows immediately or return a polling continuation, which the SDK follows
through the existing query handle and reader APIs. The same behavior
applies with and without parameters.
Both initial transports share direct-stream lifecycle handling. Handoffs
are explicitly negotiated; existing servers can continue returning
ordinary streams. Add coverage for streaming, deferred results,
cancellation, malformed decisions, and legacy responses.
```mermaid
flowchart TD
A[execute_query or execute_query_async] --> B{Parameters supplied?}
B -->|No| C[DoGet: SQL command ticket]
B -->|Yes| D[DoExchange: SQL and Arrow parameter row]
C --> E[Server chooses execution path]
D --> E
E -->|Stream| F[Initial response: streaming decision and Arrow batches]
E -->|Defer| G[Initial response: PollInfo continuation]
G --> H[SDK follows PollFlightInfo]
H -->|Still running| H
H -->|Result endpoints available| I[SDK fetches tickets with DoGet]
F --> J[Query reader yields Arrow batches]
I --> J
```
`execute_query` opens the reader; `execute_query_async` returns its
query handle. The caller does not select the transport or execution
path.
This commit is contained in:
@@ -221,10 +221,19 @@ The connected database and `default_namespace_path=["public"]` are used for
|
||||
unqualified tables. Fully qualified references can still query other databases
|
||||
and namespaces available to the same deployment. `execute_query` returns a
|
||||
reader as soon as its initial result stream is available. `execute_query_async`
|
||||
returns a query handle immediately; use it to inspect progress, open a reader,
|
||||
returns a query handle once submission is accepted; use it to inspect progress, open a reader,
|
||||
or cancel the query. The SQL client is initialized by the first query and
|
||||
retained for the lifetime of the remote connection. Query ids are random,
|
||||
connection-scoped references rather than encoded SQL or durable resume tokens:
|
||||
connection-scoped references rather than encoded SQL or durable resume tokens.
|
||||
|
||||
Both APIs start with one Flight call. The server can stream results on that
|
||||
call or return a polling continuation that the SDK follows automatically.
|
||||
This behavior is the same with and without parameters. Open the reader directly;
|
||||
a directly streamed query needs its reader to make progress. Dropping an unread
|
||||
handle or its reader stops direct streaming. Deferred execution continues until
|
||||
explicitly cancelled.
|
||||
|
||||
For example:
|
||||
|
||||
```python
|
||||
import lancedb
|
||||
|
||||
@@ -82,6 +82,8 @@ In application code, load credentials from environment variables or a secrets
|
||||
manager. The [connection guide](/basics/connect#enterprise) shows that setup.
|
||||
|
||||
`execute_query()` returns a blocking Arrow reader when results start arriving.
|
||||
The server can stream rows on the first call or return result tickets that the
|
||||
SDK follows automatically, with the same behavior for parameterized queries.
|
||||
The example uses `read_all()` to collect the small result in memory. For larger
|
||||
results, consume the reader in batches. To submit a query and check its status
|
||||
separately, use `execute_query_async()`. See [Queries](/basics/query) for query
|
||||
|
||||
Reference in New Issue
Block a user