rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-08-23 06:28:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(python): clarify threading on two-CPU containers (#3807)

Browse Source 
## Summary

- document that current LanceDB releases use one compute worker without
warning on two-vCPU containers
- distinguish compute-worker tuning from storage I/O concurrency
- direct users of affected LanceDB 0.21.1 installations to upgrade and
link the current threading guidance

## Root cause

The Lance version bundled with LanceDB 0.21.1 warned whenever the
detected CPU count was less than or equal to its default two-core I/O
reservation. A two-vCPU deployment therefore emitted the warning on
every query even though falling back to one compute worker was the
intended behavior. Lance fixed that warning condition upstream in
lance-format/lance#3710, and LanceDB current main already pins a version
containing the runtime fix; the Python package documentation did not
explain the corrected behavior or the distinct thread controls.

## Validation

- `git diff --check`
- verified the linked Lance threading-model documentation returns HTTP
200

Fixes #2326

<!-- lance-gatekeeper-fix:v1 agent=9f141242416a6dbeb43be0e80404dd4d
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
This commit is contained in:
lancedb-gatefixer[bot]
2026-08-21 16:13:49 -07:00
committed by GitHub
parent a35f7044ee
commit c7cb0b9afa
1 changed files with 19 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/README.md
+19
View File
@@ -38,6 +38,25 @@ Stable releases are created about every 2 weeks. For the latest features and bug
pip install --pre --extra-index-url https://pypi.fury.io/lancedb/ lancedb
```
### Threading in CPU-limited containers
LanceDB uses separate pools for compute work and storage I/O. On a container with
two visible CPUs, current releases intentionally use one compute worker by default;
no manual configuration is needed. If every query logs an I/O core reservation
warning on a two-CPU container, upgrade from LanceDB 0.21.1 or earlier.
The two commonly tuned environment variables control different resources:
- `LANCE_CPU_THREADS` overrides the number of compute workers. One worker is the
appropriate setting for a two-CPU container when an explicit override is needed.
- `LANCE_IO_THREADS` controls concurrent storage operations, not reserved CPU
cores. Its default can be greater than the number of CPUs because I/O workers
spend much of their time waiting for storage.
Keep the defaults unless measurements show that the workload benefits from an
override. See the [Lance threading model](https://lance.org/guide/performance/#threading-model)
for the current defaults and tuning guidance.
## Usage
### Basic Example