Qichao Chu 4494eb9e56 feat: parallelize embedding computations (#2896) ...

Implement parallel execution of multiple embedding functions using
std:🧵:scope to improve performance when a table has multiple
embedding columns.

Key changes:
- Add compute_embeddings_parallel() helper method to WithEmbeddings
- Use fast path for single embeddings (no threading overhead)
- Use scoped threads for parallel execution of multiple embeddings
- Add comprehensive tests including parallelization timing verification
- Update WithEmbeddings documentation

Performance improvements:
- I/O-bound embeddings (OpenAI, Bedrock): High benefit from concurrent
API calls
- CPU-bound embeddings (sentence-transformers): Medium benefit from core
utilization
- Single embedding: No overhead (fast path)

Closes TODO on line 266 in rust/lancedb/src/embeddings.rs

2026-01-06 14:35:56 -08:00

..

lancedb

feat: parallelize embedding computations (#2896)

2026-01-06 14:35:56 -08:00

license_header.txt

ci: check license headers (#2076)

2025-01-29 08:27:07 -08:00