lancedb

mirror of https://github.com/lancedb/lancedb.git synced 2025-12-26 14:49:57 +00:00

Files

Will Jones a975cc0a94 fix: prevent duplicate data in FTS index (#728 )

This forces the user to replace the whole FTS directory when re-creating
the index, prevent duplicate data from being created. Previously, the
whole dataset was re-added to the existing index, duplicating existing
rows in the index.

This (in combination with lancedb/lance#1707) caused #726, since the
duplicate data emitted duplicate indices for `take()` and an upstream
issue caused those queries to fail.

This solution isn't ideal, since it makes the FTS index temporarily
unavailable while the index is built. In the future, we should have
multiple FTS index directories, which would allow atomic commits of new
indexes (as well as multiple indexes for different columns).

Fixes #498.
Fixes #726.

---------

Co-authored-by: Chang She <759245+changhiskhan@users.noreply.github.com>

2024-04-05 16:24:30 -07:00

test_context.py

feat: add to_list and to_pandas api's (#556 )

2024-04-05 16:22:59 -07:00

test_db.py

feat(python): expose index cache size (#655 )

2024-04-05 16:23:49 -07:00

test_e2e_remote_db.py

feat: add to_list and to_pandas api's (#556 )

2024-04-05 16:22:59 -07:00

test_embeddings_slow.py

Multi-task instructor model with quantization support & weak_lru cache for embedding function models (#612 )