docs: remove FAQ and Troubleshooting, add Deployment > Support

FAQ content is redistributed into the pages it actually belongs to: license
and Parquet-comparison notes into the Lance format page, scale guidance into
Architecture, index-necessity and refine_factor guidance into the vector
index page, the multiprocessing/fork gotcha into SDKs and SQL, and the
weak-consistency recommendation into Consistency. General query-performance
issues are dropped since Explain and Explain Analyze already cover that
ground.

The new Support page under Deployment carries the OSS Discord and Enterprise
support-email channels that used to live in Troubleshooting.
This commit is contained in:
Jack Ye
2026-09-28 17:32:45 -05:00
parent cda12a3bda
commit 21786845a7
12 changed files with 105 additions and 179 deletions
+4 -2
View File
@@ -54,7 +54,8 @@ enforces, since there is no other gatekeeper.
**What it is good at.** Development, notebooks, single-node pipelines, embedding
LanceDB inside an application, and anything where "no service to operate" is
worth more than elasticity.
worth more than elasticity. Comfortably handles millions of vectors on one
node -- disk-based indexes keep it performant without extra infrastructure.
## Distributed {#distributed}
@@ -94,7 +95,8 @@ exists, so do the things that need one: [SQL](/sql),
**What it is good at.** Many concurrent queries, continuous ingestion and
indexing, multi-team access with real permissions, and workloads too large for
one machine.
one machine -- customers regularly operate on billions of rows and tens of
terabytes.
## What changes, precisely {#differences}
+10
View File
@@ -75,6 +75,16 @@ table = reader.read_all()
does not wait for the whole query. `execute_query_async` returns a handle
instead -- see [Queries](/basics/query).
## Python multiprocessing {#multiprocessing}
<Warning>
Do not use `fork` as the start method with Python's `multiprocessing` module.
Lance is multi-threaded internally, and `fork` with alive threads is unsafe --
use `spawn` instead. See
[this discussion](https://discuss.python.org/t/concerns-regarding-deprecation-of-fork-with-alive-threads/33555)
for why.
</Warning>
## Choosing between them {#choosing}
Reach for **the SDKs** when you are writing data, managing the catalog, running
+7
View File
@@ -28,6 +28,10 @@ Being a specification rather than a product is the point. Object storage holds
the bytes, and open engines -- PyTorch, Ray, Spark, Trino, DuckDB, Polars --
read them without going through LanceDB at all.
Lance exists because columnar formats derived from Parquet were not designed
for random access on vectors: [benchmarks](https://lancedb.com/blog/benchmarking-random-access-in-lance/)
show Lance up to 1000x faster than Parquet for that access pattern.
<Card
title="Lance format documentation"
icon="/static/assets/logo/lance-logo-gray.svg"
@@ -52,6 +56,9 @@ There is nothing to run work on your behalf, so there are no server-side
[jobs](/basics/jobs), no [SQL endpoint](/basics/apis), and no shared catalog
beyond what the storage layer itself provides.
LanceDB OSS is Apache 2.0 licensed. LanceDB Enterprise is the commercial,
deployment-only product that implements the catalog specification below.
## LanceDB Enterprise implements the catalog specification {#enterprise}
<Badge color="red">Enterprise</Badge> A `db://` connection talks to a deployment
+15 -1
View File
@@ -11,7 +11,7 @@ LanceDB emits internal metrics (currently object store request counts, bytes tra
The bridge is available in the Python and TypeScript SDKs. It is a thin wrapper over LanceDB's `metrics` recorder; your application supplies and configures the OpenTelemetry SDK.
<Note>
This page covers LanceDB OSS. LanceDB Enterprise clusters emit their own Prometheus/OpenTelemetry metrics from the server side — see the [Enterprise overview](/enterprise/) for that flow.
This page covers LanceDB OSS instrumentation. LanceDB Enterprise clusters emit their own metrics from the server side — see [Enterprise monitoring](#enterprise) below.
</Note>
## What you get {#what-you-get}
@@ -102,6 +102,20 @@ A few starting points for dashboards and alerts:
- **Retryable responses:** a rising `lance_object_store_retryable_responses_total` typically means you are being throttled and should back off or shard writes.
- **In-flight requests:** a growing `lance_object_store_in_flight_requests` gauge without a matching rise in throughput indicates queueing.
## Enterprise monitoring {#enterprise}
<Badge color="red">Enterprise</Badge> A deployment emits its own metrics from
the server side, using Prometheus for collection and OpenTelemetry (OTel) to
export them with data enrichment -- no client-side instrumentation needed.
| Component | Metrics |
| --- | --- |
| Query node | RPS, query latency, error codes, slow-query count, CPU/memory utilization |
| Plan executor | SSD cache hit/miss rate, CPU/memory utilization |
[Contact support](/deployment/support) for the comprehensive metric list and
help integrating them with your monitoring dashboard.
## Where to go next {#where-to-go-next}
<Columns cols={2}>
+30
View File
@@ -0,0 +1,30 @@
---
title: "Support"
sidebarTitle: "Support"
description: "Where to get help: community Discord for OSS, dedicated support for Enterprise."
icon: circle-question
keywords: ["support", "help", "discord", "community"]
---
## LanceDB OSS {#oss}
The best place to get help is the
[Discord community](https://discord.gg/AUEWnJ7Txb), under the relevant
language channel for Python, TypeScript, or Rust. Asking in the
language-specific channel reaches both the community and the engineering team.
## LanceDB Enterprise {#enterprise}
<Badge color="red">Enterprise</Badge> Contact your support team directly at
[support@lancedb.com](mailto:support@lancedb.com) for dedicated assistance.
## What to include {#what-to-include}
A faster answer usually depends on the same handful of details:
| | |
| --- | --- |
| SDK and version | `lancedb.__version__` (Python), `package.json` (TypeScript), the crate version (Rust) |
| Connection type | embedded path, or a `db://` deployment |
| The exact error | full traceback or error message, not a paraphrase |
| A minimal repro | the smallest table schema and call that reproduces it |
+18 -15
View File
@@ -214,6 +214,22 @@
{
"source": "/sql/show",
"destination": "/sql/ddl/job"
},
{
"source": "/faq",
"destination": "/deployment/support"
},
{
"source": "/faq/faq-oss",
"destination": "/deployment/support"
},
{
"source": "/faq/faq-enterprise",
"destination": "/deployment/support"
},
{
"source": "/troubleshooting",
"destination": "/deployment/support"
}
],
"navigation": {
@@ -338,7 +354,8 @@
"deployment/storage",
"deployment/monitoring",
"enterprise/deployment/azure",
"enterprise/benchmarks"
"enterprise/benchmarks",
"deployment/support"
]
},
{
@@ -395,20 +412,6 @@
"icon": "code"
}
]
},
{
"group": "Support",
"pages": [
"troubleshooting",
{
"group": "FAQ",
"pages": [
"faq/index",
"faq/faq-oss",
"faq/faq-enterprise"
]
}
]
}
]
},
-19
View File
@@ -1,19 +0,0 @@
---
title: "LanceDB Enterprise FAQ"
sidebarTitle: "LanceDB Enterprise"
description: "Common questions about running LanceDB Enterprise."
icon: circle-question
keywords: ["faq", "enterprise"]
---
<Badge color="red">Enterprise</Badge>
This page describes a LanceDB Enterprise capability. The embedded form
of LanceDB has no server to provide it -- see
[Architecture](/architecture) for what differs between the two.
Questions specific to running a deployment are answered with LanceDB
Enterprise.
For questions about the embedded form, see the
[LanceDB OSS FAQ](/faq/faq-oss).
-90
View File
@@ -1,90 +0,0 @@
---
title: "LanceDB: Frequently Asked Questions"
sidebarTitle: "LanceDB OSS"
description: "Commonly asked questions about LanceDB OSS."
icon: "code"
mode: wide
---
This section covers some common questions and issues that you may encounter when using LanceDB.
### Is LanceDB open source? {#is-lancedb-open-source}
LanceDB OSS is a permissively licensed embedded retrieval library available under an Apache 2.0 license. We also have a LanceDB Enterprise, a commercial product that can be deployed on a private cloud or a bring-your-own-cloud (BYOC) solution. LanceDB Enterprise transforms your data lake into a high-performance multimodal lakehouse.
### What is the difference between Lance and LanceDB? {#what-is-the-difference-between-lance-and-lancedb}
[Lance](https://github.com/lancedb/lance) is a modern lakehouse format for multimodal AI. It's perfect for building search engines, feature stores and being the foundation of large-scale ML training jobs requiring high performance IO and shuffles. It also has native support for storing, querying, and inspecting deeply nested data for robotics or large blobs like images, point clouds, and more.
LanceDB is the multimodal lakehouse that's built on top of Lance, and utilizes the underlying optimized storage format to build efficient disk-based indexes that power semantic search & retrieval applications, from RAGs to QA bots to recommender systems.
### Why invent another data format instead of using Parquet? {#why-invent-another-data-format-instead-of-using-parquet}
As we mention in our talk titled "[Lance, a modern columnar data format](https://www.youtube.com/watch?v=ixpbVyrsuL8)", Parquet and other tabular formats that derive from it are rather dated (Parquet is over 10 years old), especially when it comes to random access on vectors. We needed a format that's able to handle the complex trade-offs involved in shuffling, scanning, OLAP and filtering large datasets involving vectors, and our extensive experiments with Parquet didn't yield sufficient levels of performance for modern ML. [Our benchmarks](https://lancedb.com/blog/benchmarking-random-access-in-lance/) show that Lance is up to 1000x faster than Parquet for random access, which we believe justifies our decision to create a new data format for AI.
### Why build in Rust? {#why-build-in-rust}
We believe that the Rust ecosystem has attained mainstream maturity and that Rust will form the underpinnings of large parts of the data and ML landscape in a few years. Performance, latency and reliability are paramount to a vector DB, and building in Rust allows us to iterate and release updates more rapidly due to Rust's safety guarantees. Both Lance (the data format) and LanceDB (the database) are written entirely in Rust. We also provide Python, JavaScript, and Rust client libraries to interact with the database.
### What makes LanceDB different? {#what-makes-lancedb-different}
LanceDB is among the few embedded vector DBs out there that we believe can unlock a whole new class of LLM-powered applications in the browser or via edge functions. Lance's multimodal nature allows you to store the raw data, metadata and the embeddings all at once, unlike other solutions that typically store just the embeddings and metadata.
The Lance data format that powers our storage system also provides true zero-copy access and seamless interoperability with numerous other data formats (like Pandas, Polars, Pydantic) via Apache Arrow, as well as automatic data versioning and data management without needing extra infrastructure.
### How large of a dataset can LanceDB handle? {#how-large-of-a-dataset-can-lancedb-handle}
LanceDB and its underlying data format, Lance, are built to scale to really large amounts of data. LanceDB OSS can comfortably handle millions of vectors on a single node, making it a great fit for most applications. Its disk-based indexes keep performance strong without requiring expensive infrastructure.
If you need to scale to hundreds of millions of vectors or work with terabytes of data, we recommend [LanceDB Enterprise](/enterprise). Enterprise customers regularly operate on billions of rows, backed by distributed infrastructure designed for large-scale production workloads.
### Do I need to build a vector index to run vector search? {#do-i-need-to-build-a-vector-index-to-run-vector-search}
No. LanceDB is blazing fast (due to its disk-based index) for even brute force kNN search, within reason. In our benchmarks, computing 100K pairs of 1000-dimension vectors takes less than 20ms. For small datasets of ~100K records or applications that can accept ~100ms latency, a vector index is usually not necessary.
For large-scale (>1M) or higher dimension vectors, it is beneficial to create a vector index. See the [Vector Indexes](/indexing/vector-index/) section for more details.
### How can I speed up data inserts? {#how-can-i-speed-up-data-inserts}
LanceDB auto-parallelizes large writes when you call `table.add()` with materialized
data such as `pa.Table`, `pd.DataFrame`, or `pa.dataset()`. No extra configuration
is needed — writes are automatically split into partitions of ~1M rows or 2GB.
For best results:
- **Create an empty table first**, then call `table.add()`. The `add()` path enables
automatic write parallelism, while passing data directly to `create_table()` does not.
- **For file-based data**, use `pyarrow.dataset.dataset("path/to/data/", format="parquet")`
so LanceDB can stream from disk without loading everything into memory.
- **Avoid inserting one row at a time.** Each insert creates a new data fragment on
disk. Batch your data into Arrow tables, DataFrames, or use iterators.
See [Ingesting large datasets](/ingestion/insert#large-datasets) for full examples.
### Do I need to set a refine factor when using an index? {#do-i-need-to-set-a-refine-factor-when-using-an-index}
Yes. LanceDB uses PQ, or Product Quantization, to compress vectors and speed up search when using an ANN index. However, because PQ is a lossy compression algorithm, it tends to reduce recall while also reducing the index size. To address this trade-off, we introduce a process called **refinement**. The normal process computes distances by operating on the compressed PQ vectors. The refinement factor (*rf*) is a multiplier that takes the top-k similar PQ vectors to a given query, fetches `rf * k` *full* vectors and computes the raw vector distances between them and the query vector, reordering the top-k results based on these scores instead.
For example, if you're retrieving the top 10 results and set `refine_factor` to 25, LanceDB will fetch the 250 most similar vectors (according to PQ), compute the distances again based on the full vectors for those 250 and then re-rank based on their scores. This can significantly improve recall, with a small added latency cost (typically a few milliseconds), so it's recommended you set a `refine_factor` of anywhere between 5-50 and measure its impact on latency prior to deploying your solution.
### How can I improve IVF-PQ recall while keeping latency low? {#how-can-i-improve-ivf-pq-recall-while-keeping-latency-low}
When using an IVF-PQ index, there's a trade-off between recall and latency at query time. You can improve recall by increasing the number of probes and the `refine_factor`. In our benchmark on the GIST-1M dataset, we show that it's possible to achieve >0.95 recall with a latency of under 10 ms on most systems, using ~50 probes and a `refine_factor` of 50. This is, of course, subject to the dataset at hand and a quick sensitivity study can be performed on your own data. You can find more details on the benchmark in a past [blog post](https://medium.com/etoai/benchmarking-lancedb-92b01032874a).
![](/static/assets/images/faq/recall-vs-latency.webp)
### How much data can LanceDB practically manage without affecting performance? {#how-much-data-can-lancedb-practically-manage-without-affecting-performance}
We target good performance on ~10-50 billion rows and ~10-30 TB of data. For the best performance and
scalability guarantees, check out [LanceDB Enterprise](/enterprise).
### Does LanceDB support concurrent operations? {#does-lancedb-support-concurrent-operations}
LanceDB can handle concurrent reads very well, and can scale horizontally. The main constraint is how well the storage layer you've chosen, scales. For writes, we support concurrent writing, though too many concurrent writers can lead to failing writes as there is a limited number of times a writer retries a commit.
<Warning>
If you use Python's multiprocessing, you should probably not use `fork` as Lance is multi-threaded
internally and `fork` and multi-threaded Python do not work well together.
[Refer to this discussion](https://discuss.python.org/t/concerns-regarding-deprecation-of-fork-with-alive-threads/33555)
for more information.
</Warning>
-17
View File
@@ -1,17 +0,0 @@
---
title: Frequently Asked Questions
sidebarTitle: "Overview"
description: "Common questions about LanceDB"
icon: "question-circle"
---
Find answers to common questions about LanceDB across different deployment options and use cases.
<Card title="Can't find what you're looking for?" icon="question">
Reach out on [Discord](https://discord.gg/AUEWnJ7Txb) for community support or [contact us](mailto:support@lancedb.com) for Enterprise assistance.
</Card>
| Category | Description |
|:---------|:------------|
| [LanceDB OSS](/faq/faq-oss) | Questions about LanceDB open source deployment, installation, and community support |
| [LanceDB Enterprise](/faq/faq-enterprise) | Questions about LanceDB Enterprise features, security, compliance, and support |
+14 -1
View File
@@ -29,6 +29,18 @@ import {
Vector indexes are robust tools in facilitating fast searches across large numeric datasets.
LanceDB implements **ANN (Approximate Nearest-Neighbor)** queries with several techniques that provide benefits across a variety of use cases.
## Do you need one? {#do-you-need-one}
Not always. LanceDB's disk-based brute-force kNN search is fast even without an
index -- computing 100K pairs of 1000-dimension vectors takes under 20ms in
benchmarks. For datasets around 100K rows or fewer, or applications that can
accept ~100ms latency, a vector index is usually unnecessary overhead.
Build one once a table passes roughly a million rows, dimensionality is high,
or query latency matters more than that. See
[`bypass_vector_index`](#search-time-parameters) to run an exact search
side-by-side and measure whether an index is worth it for your data.
## Choosing the Right Index
LanceDB vector indexes are combined with several [quantization](/indexing/vector-index) techniques to admit efficient storage.
@@ -131,7 +143,7 @@ Learn how to configure, build, and search LanceDB vector indexes, including buil
| `minimum_nprobes` | Minimum number of partitions scanned. |
| `maximum_nprobes` | Maximum number of partitions scanned. Only scans more than `minimum_nprobes` if an initial pass does not return enough results — useful for narrow filters. Set to `0` to remove the cap. |
| `ef` | HNSW search-time exploration factor. Start around `1.5 * k` and increase up to `10 * k` for higher recall. |
| `refine_factor` | Reads and reranks additional candidates in memory to recover recall lost to quantization. |
| `refine_factor` | Reads and reranks additional candidates in memory to recover recall lost to quantization. Start between `5` and `50` (a factor of `25` reads and reranks the top `25 * k` candidates) and measure the latency cost, typically a few milliseconds, against the recall gained. |
| `distance_range(lower_bound, upper_bound)` | Return only rows whose distance falls within `[lower_bound, upper_bound)`. Either bound is optional. Useful for near-duplicate detection or "close-enough" matching. |
| `bypass_vector_index()` | Ignore the ANN index entirely and perform an exact (flat) scan. Can be used to measure ANN `recall@k`, or to query with a metric the index was not built for (e.g., a non-cosine query on a multivector column). |
@@ -254,6 +266,7 @@ The snippet below builds and queries an `IVF_HNSW_SQ` index.
In LanceDB Enterprise, vector indexes are managed **automatically**. The system asynchronously updates and optimizes indexes as a background process:
- Automatically manages indexing parameters and storage
- Infers vector columns from the schema
- GPU-based indexing is available on request -- [contact support](/deployment/support) to enable it for your deployment. A few billion vectors index in under 4 hours on GPU.
<Badge color="gray">Open-Source</Badge>
LanceDB OSS users can manually create vector indexes by calling `table.create_index()`.
+7
View File
@@ -60,6 +60,13 @@ of refreshing against storage, which raises per-read latency and cost.
In <Badge color="red">Enterprise</Badge> deployments, the server-side default freshness is still
controlled by the cluster-level `weak_read_consistency_interval_seconds` parameter; the client setting
tightens that bound on a per-connection basis.
<Tip>
30-60 seconds is a reasonable starting point for
`weak_read_consistency_interval_seconds`: it cuts unnecessary object-store
checks while keeping data fresh enough for most applications. This setting
only affects reads -- writes always remain strongly consistent.
</Tip>
</Callout>
## Configure Consistency Parameters {#configure-consistency-parameters}
-34
View File
@@ -1,34 +0,0 @@
---
title: "Troubleshooting"
sidebarTitle: "Troubleshooting"
description: "Tips for troubleshooting basic LanceDB issues."
icon: "tools"
---
## Frequently-asked questions {#frequently-asked-questions}
For commonly asked questions about LanceDB, please refer to our [FAQ section](/faq).
## Getting technical support {#getting-technical-support}
If you're using LanceDB OSS, the best place to get help is in our
[Discord community](https://discord.gg/AUEWnJ7Txb),
under the relevant language channel for Python, TypeScript, or Rust.
By asking in the language-specific channel, you can get help from the community
and our engineering team.
If you are a LanceDB Enterprise user, please contact our support team at [support@lancedb.com](mailto:support@lancedb.com) for dedicated assistance.
## General issues {#general-issues}
### Slow or unexpected query results {#slow-or-unexpected-query-results}
If you have slow queries or unexpected query results, it can be helpful to
print the resolved query plan.
LanceDB provides two powerful tools for query analysis and optimization: `explain_plan` and `analyze_plan`.
Read the full guides on [Explain](/search/explain) and [Explain Analyze](/search/explain).
### The Python multiprocessing module {#the-python-multiprocessing-module}
Multiprocessing with `fork` is not supported. You should use `spawn` instead.