rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-01-05 19:32:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,501 Commits 115 Branches 397 Tags
b66cd943a7c2d8db1d587a52b61b7e722602c0bc
Commit Graph

574 Commits

Author SHA1 Message Date
Lei Xu
5857cb4c6e docs: add a section to describe scalar index (#1495) 2024-08-16 18:48:29 -07:00
BubbleCal
0fa50775d6 feat: support to query/index FTS on RemoteTable/AsyncTable (#1537) 
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
 2024-08-16 12:01:05 +08:00
Gagan Bhullar
20faa4424b feat(python): add delete unverified parameter (#1542) 
PR fixes #1527
 2024-08-15 09:01:32 -07:00
BubbleCal
b624fc59eb docs: add create_fts_index doc in Python API Reference (#1533) 
resolve #1313

---------

Signed-off-by: BubbleCal <bubble-cal@outlook.com>
 2024-08-15 11:35:16 +08:00
BubbleCal
501817cfac chore: bump the required python version to 3.9 (#1541) 
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
 2024-08-14 08:44:31 -07:00
Ryan Green
b3daa25f46 feat: allow new scalar index types to be created in remote table (#1538) 2024-08-13 16:05:42 -02:30
Lance Release
ff5bbfdd4c Bump version: 0.12.0 → 0.13.0-beta.0 2024-08-12 19:47:57 +00:00
Lei Xu
b2317c904d feat: create bitmap and label list scalar index using python async api (#1529) 
* Expose `bitmap` and `LabelList` scalar index type via Rust and Async
Python API
* Add documents
 2024-08-11 09:16:11 -07:00
BubbleCal
613f3063b9 chore: upgrade lance to 0.16.1 (#1524) 
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
 2024-08-09 19:18:05 +08:00
Gagan Bhullar
9c1adff426 feat(python): add to_list to async api (#1520) 
PR fixes #1517
 2024-08-08 11:45:20 -07:00
BubbleCal
f9d5fa88a1 feat!: migrate FTS from tantivy to lance-index (#1483) 
Lance now supports FTS, so add it into lancedb Python, TypeScript and
Rust SDKs.

For Python, we still use tantivy based FTS by default because the lance
FTS index now misses some features of tantivy.

For Python:
- Support to create lance based FTS index
- Support to specify columns for full text search (only available for
lance based FTS index)

For TypeScript:
- Change the search method so that it can accept both string and vector
- Support full text search

For Rust
- Support full text search

The others:
- Update the FTS doc

BREAKING CHANGE: 
- for Python, this renames the attached score column of FTS from "score"
to "_score", this could be a breaking change for users that rely the
scores

---------

Signed-off-by: BubbleCal <bubble-cal@outlook.com>
 2024-08-08 15:33:15 +08:00
Lance Release
ec39d98571 Bump version: 0.12.0-beta.0 → 0.12.0 2024-08-07 20:55:40 +00:00
Lance Release
0cb37f0e5e Bump version: 0.11.0 → 0.12.0-beta.0 2024-08-07 20:55:39 +00:00
Lei Xu
2bdf0a02f9 feat!: upgrade lance to 0.16 (#1519) 2024-08-07 13:15:22 -07:00
Gagan Bhullar
32123713fd feat(python): optimize stats repr method (#1510) 
PR fixes #1507
 2024-08-07 08:47:52 -07:00
Gagan Bhullar
d5a01ffe7b feat(python): index config repr method (#1509) 
PR fixes #1506
 2024-08-07 08:46:46 -07:00
Ayush Chaurasia
e01045692c feat(python): support embedding functions in remote table (#1405) 2024-08-07 20:22:43 +05:30
Ayush Chaurasia
4769d8eb76 feat(python): multi-vector reranking support (#1481) 
Currently targeting the following usage:
```
from lancedb.rerankers import CrossEncoderReranker

reranker = CrossEncoderReranker()

query = "hello"

res1 = table.search(query, vector_column_name="vector").limit(3)
res2 = table.search(query, vector_column_name="text_vector").limit(3)
res3 = table.search(query, vector_column_name="meta_vector").limit(3)

reranked = reranker.rerank_multivector(
               [res1, res2, res3],  
              deduplicate=True,
              query=query # some reranker models need query
)
```
- This implements rerank_multivector function in the base reranker so
that all rerankers that implement rerank_vector will automatically have
multivector reranking support
- Special case for RRF reranker that just uses its existing
rerank_hybrid fcn to multi-vector reranking.

---------

Co-authored-by: Weston Pace <weston.pace@gmail.com>
 2024-08-07 01:45:46 +05:30
Ayush Chaurasia
d07d7a5980 chore: update polars version range (#1508) 2024-08-06 23:43:15 +05:30
Robby
8d2ff7b210 feat(python): add watsonx embeddings to registry (#1486) 
Related issue: https://github.com/lancedb/lancedb/issues/1412

---------

Co-authored-by: Robby <h0rv@users.noreply.github.com>
 2024-08-06 10:58:33 +05:30
Ryan Green
6af69b57ad fix: return LanceMergeInsertBuilder in overridden merge_insert method on remote table (#1484) 2024-07-31 12:25:16 -02:30
Lance Release
7b6d3f943b Bump version: 0.11.0-beta.0 → 0.11.0 2024-07-26 20:18:31 +00:00
Lance Release
676876f4d5 Bump version: 0.10.2 → 0.11.0-beta.0 2024-07-26 20:18:30 +00:00
Will Jones
9555efacf9 feat: upgrade lance to 0.15.0 (#1477) 
Changelog: https://github.com/lancedb/lance/releases/tag/v0.15.0

* Fixes #1466
* Closes #1475
* Fixes #1446
 2024-07-26 09:13:49 -07:00
Chang She
374c1e7aba fix: infer schema from huggingface dataset (#1444) 
Closes #1383

When creating a table from a HuggingFace dataset, infer the arrow schema
directly
 2024-07-23 13:12:34 -07:00
Ayush Chaurasia
0255221086 feat: add reciprocal rank fusion reranker (#1456) 
Implements https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf

Refactors the hybrid search only rerrankers test to avoid repetition.
 2024-07-23 21:37:17 +05:30
Lance Release
1d5da1d069 Bump version: 0.10.2-beta.0 → 0.10.2 2024-07-23 13:48:48 +00:00
Lance Release
0c0ec1c404 Bump version: 0.10.1 → 0.10.2-beta.0 2024-07-23 13:48:47 +00:00
Weston Pace
d4aad82aec fix: don't use v2 by default on empty table (#1469) 2024-07-23 06:47:49 -07:00
Will Jones
4f601a2d4c fix: handle camelCase column names in select (#1460) 
Fixes #1385
 2024-07-22 12:53:17 -07:00
Lei Xu
c9c61eb060 docs: expose merge_insert doc for remote python SDK (#1464) 
`merge_insert` API is not shown up on
[`RemoteTable`](https://lancedb.github.io/lancedb/python/saas-python/#lancedb.remote.table.RemoteTable)
today

* Also bump `ruff` version as well
 2024-07-22 10:48:16 -07:00
Ayush Chaurasia
ed7bd45c17 chore: choose appropriate args for concat_table based on pyarrow version & refactor reranker tests (#1455) 2024-07-18 21:04:59 +05:30
Magnus
dc609a337d fix: added support for trust_remote_code (#1454) 
Closes #1285 

Added trust_remote_code to the SentenceTransformerEmbeddings class.
Defaults to `False`
 2024-07-18 19:37:52 +05:30
Lance Release
2c36767f20 Bump version: 0.10.1-beta.0 → 0.10.1 2024-07-17 14:04:40 +00:00
Lance Release
1fa7e96aa1 Bump version: 0.10.0 → 0.10.1-beta.0 2024-07-17 14:04:39 +00:00
Lance Release
11959cc5d6 Bump version: 0.10.0-beta.0 → 0.10.0 2024-07-13 08:55:22 +00:00
Lance Release
7c65cec8d7 Bump version: 0.9.0 → 0.10.0-beta.0 2024-07-13 08:55:22 +00:00
Adam Azzam
82621d5b13 chore: typing for lance.connect (#1441) 
Feel free to close if this is a distraction, but untyped keywords in
lance.connect is throwing pylance errors in strict mode.

<img width="683" alt="Screenshot 2024-07-11 at 1 21 04 PM"
src="https://github.com/lancedb/lancedb/assets/33043305/fe6cd4d9-4e59-413d-87f2-aabb9ff84cc4">
 2024-07-12 10:39:28 -07:00
Lei Xu
0708428357 feat: support update over binary field (#1440) 2024-07-12 09:22:00 -07:00
BubbleCal
137d86d3c5 chore: bump lance to 0.14.1 (#1442) 
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
 2024-07-12 21:41:59 +08:00
Joan Fontanals
cef24801f4 docs: add jina reranker to index (#1427) 
PR to add JinaReranker documentation page to the rerankers index
 2024-07-09 14:39:35 +05:30
forrestmckee
b4436e0804 refactor: update type hint and remove unused import (#1436) 
change typehint on `_invert_score` from `List[float]` to `float`. remove
unnecessary typing import
 2024-07-09 13:56:45 +05:30
Lei Xu
fd5ca20f34 chore: bump lance to 0.14 (#1430) 2024-07-06 14:10:42 -07:00
Joan Fontanals
08d25c5a80 feat: add Jina integration in Python for Embedding and Reranker (#1424) 
Integration of Jina Embeddings and Rerankers through its API
 2024-07-05 01:34:43 +05:30
Nuvic
46c6ff889d feat: add the explain_plan function (#1328) 
It's useful to see the underlying query plan for debugging purposes.
This exposes LanceScanner's `explain_plan` function. Addresses #1288

---------

Co-authored-by: Will Jones <willjones127@gmail.com>
 2024-07-02 11:10:01 -07:00
BubbleCal
12b3c87964 feat: support to create more vector index types (#1407) 
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
 2024-07-02 10:53:03 -02:30
Will Jones
865ed99881 feat: dynamodb commit store support (#1410) 
This allows users to specify URIs like:

```
s3+ddb://my_bucket/path?ddbTableName=myCommitTable
```

and it will support concurrent writes in S3.

* [x] Add dynamodb integration tests
* [x] Add modifications to get it working in Python sync API
* [x] Added section in documentation describing how to configure.

Closes #534

---------

Co-authored-by: universalmind303 <cory.grinstead@gmail.com>
 2024-06-28 09:30:36 -07:00
Lance Release
bc0814767b Bump version: 0.9.0-beta.0 → 0.9.0 2024-06-25 00:25:27 +00:00
Lance Release
8960a8e535 Bump version: 0.8.2 → 0.9.0-beta.0 2024-06-25 00:25:27 +00:00
Weston Pace
a8568ddc72 feat: upgrade to lance 0.13.0 (#1404) 2024-06-24 17:22:57 -07:00