mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-14 22:56:54 +00:00
Xuanwo
9b13e8f725
Python remote queries materialize Blob v2 payloads with `blob_mode="bytes"` through synchronous and asynchronous `to_arrow`, `to_list`, `to_pandas`, and `to_batches`. Deferred reads bind to the actual producing dataset without changing the caller's live table handle, including concurrent writes and same-name DROP/CREATE. Trust only the authoritative `x-lancedb-query-version` plus `x-lancedb-query-snapshot`. A numeric version alone cannot distinguish a recreated table; the ordinary live watermark and user-supplied Arrow metadata cannot establish provenance. An independent immutable handle carries the original identity through every batch, byte-limit split, and large-Blob Range fallback, and requires a verified response echo before accepting payloads. Aliases, NULL/empty results, rerankers, and hybrid legs retain consistent provenance. Missing capability, mixed identities, or a serving node ignoring the precondition fails explicitly; ordinary descriptor queries remain available on older servers. The source docs distinguish Enterprise SQL payload bytes from SDK descriptors. Server dependency: https://github.com/lancedb/sophon/pull/10211. Validation uses retained, independently audited local evidence: 724 Python cases passed (one optional PyAV skip), 67 focused SDK Rust cases passed, 36 server checks passed, and 15 actual REST/SDK scenarios passed across retained runs. These cover sync/async and Arrow file/stream recreation, concurrent overwrite, deletion of a selected version, legacy capabilities, forged IPC metadata, WAL exclusion, and actual mid-batch recreation. The original aggregate experiment exited 101 because its extra harness assumed a Python batch size would split the native result; its 36 server checks and 13 completed scenarios remain valid. The corrected native-batch experiment passed separately for sync and async. No aggregate all-green claim is made. The public batched-range operation has separate cross-partition identity integration in https://github.com/lancedb/sophon/pull/10219 and https://github.com/lancedb/lancedb/pull/4994. This change does not claim to fix that loop. Realtime/MemWAL remains outside Blob v2 GA, without disabling existing writes or compaction. No dependency pins changed. Merge/rollout hold remains: upgrade every serving node before enabling the new SDK capability, verify proxy forwarding of both authoritative headers and the verification echo, and preserve queried versions for later reads. Missing or inconsistent capability fails explicitly; identity does not retain versions or authenticate storage paths. Actual deployed proxy/Cloud/PE behavior and production retention remain unverified. Old SDKs do not gain incarnation protection from a server-only upgrade. Published documentation retains the Early access badge and released Enterprise instructions, including explicit fetches for copying. The new Enterprise `blob_mode="bytes"` guidance, snapshot-bound copy example, future streaming restrictions and GA wording remain source-only MDX drafts until release. The correction distinguishing released Enterprise SQL bytes from SDK descriptors remains visible. Documentation validation checked comment boundaries, visible instructions, Python example syntax and whitespace; the site was not built. This documentation-only revision reuses the implementation evidence above.
The Multimodal Lakehouse for AI
How to Install ✦ Detailed Documentation ✦ Tutorials and Recipes ✦ Contributors
The ultimate multimodal data platform for AI/ML applications.
LanceDB is designed for fast, scalable, and production-ready vector search. It is built on top of the Lance columnar format. You can store, index, and search over petabytes of multimodal data and vectors with ease. LanceDB is a central location where developers can build, train and analyze their AI workloads.
Demo: Multimodal Search by Keyword, Vector or with SQL
Star LanceDB to get updates!
⭐ Click here ⭐ to see how fast we're growing!
Key Features:
- Fast Vector Search: Search billions of vectors in milliseconds with state-of-the-art indexing.
- Comprehensive Search: Support for vector similarity search, full-text search and SQL.
- Multimodal Support: Store, query and filter vectors, metadata and multimodal data (text, images, videos, point clouds, and more).
- Advanced Features: Zero-copy, automatic versioning, manage versions of your data without needing extra infrastructure. GPU support in building vector index.
Products:
- LanceDB Embedded (open source): 100% open source, runs in your application's process; your data can live on local disk or in object storage. No vendor lock-in.
- LanceDB Enterprise: Production-scale distributed search, indexing, and maintenance. Plus, complete data sovereignty and security.
Ecosystem:
- Columnar Storage: Built on the Lance columnar format for efficient storage and analytics.
- Seamless Integration: Python, Node.js, Rust, and REST APIs for easy integration. Native Python and Javascript/Typescript support.
- Rich Ecosystem: Integrations with LangChain 🦜️🔗, LlamaIndex 🦙, Apache-Arrow, Pandas, Polars, DuckDB and more on the way.
How to Install:
Follow the Quickstart doc to set up LanceDB locally.
API & SDK: We also support Python, Typescript and Rust SDKs
| Interface | Documentation |
|---|---|
| Python SDK | https://lancedb.github.io/lancedb/python/python/ |
| Typescript SDK | https://lancedb.github.io/lancedb/js/globals/ |
| Rust SDK | https://docs.rs/lancedb/latest/lancedb/index.html |
| REST API | https://docs.lancedb.com/api-reference/rest |
Join Us and Contribute
We welcome contributions from everyone! Whether you're a developer, researcher, or just someone who wants to help out.
If you have any suggestions or feature requests, please feel free to open an issue on GitHub or discuss it on our Discord server.
Check out the GitHub Issues if you would like to work on the features that are planned for the future. If you have any suggestions or feature requests, please feel free to open an issue on GitHub.
Contributors
Stay in Touch With Us