mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 14:16:54 +00:00
Xuanwo
9b13e8f725
Python remote queries materialize Blob v2 payloads with `blob_mode="bytes"` through synchronous and asynchronous `to_arrow`, `to_list`, `to_pandas`, and `to_batches`. Deferred reads bind to the actual producing dataset without changing the caller's live table handle, including concurrent writes and same-name DROP/CREATE. Trust only the authoritative `x-lancedb-query-version` plus `x-lancedb-query-snapshot`. A numeric version alone cannot distinguish a recreated table; the ordinary live watermark and user-supplied Arrow metadata cannot establish provenance. An independent immutable handle carries the original identity through every batch, byte-limit split, and large-Blob Range fallback, and requires a verified response echo before accepting payloads. Aliases, NULL/empty results, rerankers, and hybrid legs retain consistent provenance. Missing capability, mixed identities, or a serving node ignoring the precondition fails explicitly; ordinary descriptor queries remain available on older servers. The source docs distinguish Enterprise SQL payload bytes from SDK descriptors. Server dependency: https://github.com/lancedb/sophon/pull/10211. Validation uses retained, independently audited local evidence: 724 Python cases passed (one optional PyAV skip), 67 focused SDK Rust cases passed, 36 server checks passed, and 15 actual REST/SDK scenarios passed across retained runs. These cover sync/async and Arrow file/stream recreation, concurrent overwrite, deletion of a selected version, legacy capabilities, forged IPC metadata, WAL exclusion, and actual mid-batch recreation. The original aggregate experiment exited 101 because its extra harness assumed a Python batch size would split the native result; its 36 server checks and 13 completed scenarios remain valid. The corrected native-batch experiment passed separately for sync and async. No aggregate all-green claim is made. The public batched-range operation has separate cross-partition identity integration in https://github.com/lancedb/sophon/pull/10219 and https://github.com/lancedb/lancedb/pull/4994. This change does not claim to fix that loop. Realtime/MemWAL remains outside Blob v2 GA, without disabling existing writes or compaction. No dependency pins changed. Merge/rollout hold remains: upgrade every serving node before enabling the new SDK capability, verify proxy forwarding of both authoritative headers and the verification echo, and preserve queried versions for later reads. Missing or inconsistent capability fails explicitly; identity does not retain versions or authenticate storage paths. Actual deployed proxy/Cloud/PE behavior and production retention remain unverified. Old SDKs do not gain incarnation protection from a server-only upgrade. Published documentation retains the Early access badge and released Enterprise instructions, including explicit fetches for copying. The new Enterprise `blob_mode="bytes"` guidance, snapshot-bound copy example, future streaming restrictions and GA wording remain source-only MDX drafts until release. The correction distinguishing released Enterprise SQL bytes from SDK descriptors remains visible. Documentation validation checked comment boundaries, visible instructions, Python example syntax and whitespace; the site was not built. This documentation-only revision reuses the implementation evidence above.