Ayush Chaurasia
dc72ece847
feat!: better api for manual hybrid queries ( #1575 )
...
Currently, the only documented way of performing hybrid search is by
using embedding API and passing string queries that get automatically
embedded. There are use cases where users might like to pass vectors and
text manually instead.
This ticket contains more information and historical context -
https://github.com/lancedb/lancedb/issues/937
This breaks a undocumented pathway that allowed passing (vector, text)
tuple queries which was intended to be temporary, so this is marked as a
breaking change. For all practical purposes, this should not really
impact most users
### usage
```
results = table.search(query_type="hybrid")
.vector(vector_query)
.text(text_query)
.limit(5)
.to_pandas()
```
2024-08-30 17:37:58 +05:30
BubbleCal
1521435193
fix: specify column to search for FTS ( #1572 )
...
Before this we ignored the `fts_columns` parameter, and for now we
support to search on only one column, it could lead to an error if we
have multiple indexed columns for FTS
---------
Signed-off-by: BubbleCal <bubble-cal@outlook.com >
2024-08-29 23:43:46 +08:00
Ayush Chaurasia
bfe8fccfab
docs: add hnsw docs ( #1570 )
2024-08-29 15:16:27 +05:30
Rithik Kumar
6f6eb170a9
docs: revamp Python example: Overview page and remove redundant examples and notebooks ( #1574 )
...
before:
After:
---------
Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com >
2024-08-29 13:48:10 +05:30
Rithik Kumar
dd1c16bbaf
docs: fix links, convert backslash to forward slash in mkdocs.yml ( #1571 )
...
Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com >
2024-08-28 16:07:57 +05:30
Gagan Bhullar
a76186ee83
fix(node): read consistency level fix ( #1567 )
...
PR fixes #1565
2024-08-27 17:03:42 -07:00
Rithik Kumar
ae85008714
docs: revamp embedding models ( #1568 )
...
before:
After:
---------
Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com >
2024-08-27 17:14:35 +05:30
Gagan Bhullar
a85f039352
fix(bug): limit fix ( #1548 )
...
PR fixes #1151
2024-08-26 14:25:14 -07:00
Bill Chambers
9c25998110
docs: update serverless_lancedb_with_s3_and_lambda.md ( #1559 )
2024-08-26 14:55:28 +05:30
Ayush Chaurasia
549ca51a8a
feat: add answerdotai rerankers support and minor improvements ( #1560 )
...
This PR:
- Adds missing license headers
- Integrates with answerdotai Rerankers package
- Updates ColbertReranker to subclass answerdotai package. This is done
to keep backwards compatibility as some users might be used to importing
ColbertReranker directly
- Set `trust_remote_code` to ` True` by default in CrossEncoder and
sentence-transformer based rerankers
2024-08-26 13:25:10 +05:30
Rithik Kumar
632007d0e2
docs: add recommender system example ( #1561 )
...
before:
After:
---------
Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com >
2024-08-25 12:30:30 +05:30
Lance Release
02d85a4ea4
Updating package-lock.json
2024-08-23 13:56:54 +00:00
Lance Release
a9d0625e2b
Bump version: 0.10.0-beta.0 → 0.10.0-beta.1
2024-08-23 13:56:34 +00:00
Lance Release
89bcc1b2e7
Bump version: 0.13.0-beta.0 → 0.13.0-beta.1
2024-08-23 13:56:30 +00:00
rahuljo
6ad5553eca
docs: add dlt-lancedb integration page ( #1551 )
...
Co-authored-by: Akela Drissner-Schmid <32450038+akelad@users.noreply.github.com >
2024-08-22 15:18:49 +05:30
Gagan Bhullar
6eb7ccfdee
fix: rerank attribute unknown ( #1554 )
...
PR fixes #1550
2024-08-22 11:46:36 +05:30
Rithik Kumar
758c82858f
docs: add AI agent example ( #1553 )
...
before:
After:
---------
Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com >
2024-08-22 00:54:05 +05:30
Rithik Kumar
0cbc9cd551
docs: add evaluation example ( #1552 )
...
before:
After:
---------
Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com >
2024-08-21 20:37:04 +05:30
Ayush Chaurasia
7d65dd97cf
chore(python): update Colbert architecture and minor improvements ( #1547 )
...
- Update ColBertReranker architecture: The current implementation
doesn't use the right arch. This PR uses the implementation in Rerankers
library. Fixes https://github.com/lancedb/lancedb/issues/1546
Benchmark diff (hit rate):
Hybrid - 91 vs 87
reranked vector - 85 vs 80
- Reranking in FTS is basically disabled in main after last week's FTS
updates. I think there's no blocker in supporting that?
- Allow overriding accelerators: Most transformer based Rerankers and
Embedding automatically select device. This PR allows overriding those
settings by passing `device`. Fixes:
https://github.com/lancedb/lancedb/issues/1487
---------
Co-authored-by: BubbleCal <bubble-cal@outlook.com >
2024-08-21 12:26:52 +05:30
Ayush Chaurasia
85bb7e54e4
docs: missing griffe dependency for mkdocs deployment ( #1545 )
2024-08-19 07:48:23 +05:30
Rithik Kumar
21014cab45
docs: add chatbot example and improve quality of other examples ( #1544 )
2024-08-17 12:35:33 +05:30
Lei Xu
5857cb4c6e
docs: add a section to describe scalar index ( #1495 )
2024-08-16 18:48:29 -07:00
Rithik Kumar
09ce6c5bb5
docs: add vector search example ( #1543 )
2024-08-16 21:30:45 +05:30
BubbleCal
0fa50775d6
feat: support to query/index FTS on RemoteTable/AsyncTable ( #1537 )
...
Signed-off-by: BubbleCal <bubble-cal@outlook.com >
2024-08-16 12:01:05 +08:00
Gagan Bhullar
20faa4424b
feat(python): add delete unverified parameter ( #1542 )
...
PR fixes #1527
2024-08-15 09:01:32 -07:00
BubbleCal
b624fc59eb
docs: add create_fts_index doc in Python API Reference ( #1533 )
...
resolve #1313
---------
Signed-off-by: BubbleCal <bubble-cal@outlook.com >
2024-08-15 11:35:16 +08:00
Gagan Bhullar
d2caa5e202
feat(nodejs): add delete unverified ( #1530 )
...
PR fixes part of #1527
2024-08-14 08:53:53 -07:00
BubbleCal
501817cfac
chore: bump the required python version to 3.9 ( #1541 )
...
Signed-off-by: BubbleCal <bubble-cal@outlook.com >
2024-08-14 08:44:31 -07:00
Ryan Green
b3daa25f46
feat: allow new scalar index types to be created in remote table ( #1538 )
2024-08-13 16:05:42 -02:30
Matt Basta
6008a8257b
fix: remove native.d.ts from .npmignore ( #1531 )
...
This removes the type definitions for a number of important TypeScript
interfaces from `.npmignore` so that the package is not incorrectly
typed `any` in a number of places.
---
Presently the `opts` argument to `lancedb.connect` is typed `any`, even
though it shouldn't be.
<img width="560" alt="image"
src="https://github.com/user-attachments/assets/5c974ce8-5a59-44a1-935d-cbb808f0ea24 ">
Clicking into the type definitions for the published package, it has the
correct type signature:
<img width="831" alt="image"
src="https://github.com/user-attachments/assets/6e39a519-13ff-4ca8-95ae-85538ac59d5d ">
However, `ConnectionOptions` is imported from `native.js` (along with a
number of other imports a bit further down):
<img width="384" alt="image"
src="https://github.com/user-attachments/assets/10c1b055-ae78-4088-922e-2816af64c23c ">
This is not otherwise an issue, except that the type definitions for
`native.js` are not included in the published package:
<img width="217" alt="image"
src="https://github.com/user-attachments/assets/f15cd3b6-a8de-4011-9fa2-391858da20ec ">
I haven't compiled the Rust code and run the build script, but I
strongly suspect that disincluding the type definitions in `.npmignore`
is ultimately the root cause here.
2024-08-13 10:06:15 -07:00
Lance Release
aaff43d304
Updating package-lock.json
2024-08-12 19:48:18 +00:00
Lance Release
d4c3a8ca87
Bump version: 0.9.0 → 0.10.0-beta.0
2024-08-12 19:48:02 +00:00
Lance Release
ff5bbfdd4c
Bump version: 0.12.0 → 0.13.0-beta.0
2024-08-12 19:47:57 +00:00
Lei Xu
694ca30c7c
feat(nodejs): add bitmap and label list index types in nodejs ( #1532 )
2024-08-11 12:06:02 -07:00
Lei Xu
b2317c904d
feat: create bitmap and label list scalar index using python async api ( #1529 )
...
* Expose `bitmap` and `LabelList` scalar index type via Rust and Async
Python API
* Add documents
2024-08-11 09:16:11 -07:00
BubbleCal
613f3063b9
chore: upgrade lance to 0.16.1 ( #1524 )
...
Signed-off-by: BubbleCal <bubble-cal@outlook.com >
2024-08-09 19:18:05 +08:00
BubbleCal
5d2cd7fb2e
chore: upgrade object_store to 0.10.2 ( #1523 )
...
To use the same version with lance
Signed-off-by: BubbleCal <bubble-cal@outlook.com >
2024-08-09 12:03:46 +08:00
Ayush Chaurasia
a88e9bb134
docs: add lancedb embedding fcn on cloud docs ( #1521 )
2024-08-09 07:21:04 +05:30
Gagan Bhullar
9c1adff426
feat(python): add to_list to async api ( #1520 )
...
PR fixes #1517
2024-08-08 11:45:20 -07:00
BubbleCal
f9d5fa88a1
feat!: migrate FTS from tantivy to lance-index ( #1483 )
...
Lance now supports FTS, so add it into lancedb Python, TypeScript and
Rust SDKs.
For Python, we still use tantivy based FTS by default because the lance
FTS index now misses some features of tantivy.
For Python:
- Support to create lance based FTS index
- Support to specify columns for full text search (only available for
lance based FTS index)
For TypeScript:
- Change the search method so that it can accept both string and vector
- Support full text search
For Rust
- Support full text search
The others:
- Update the FTS doc
BREAKING CHANGE:
- for Python, this renames the attached score column of FTS from "score"
to "_score", this could be a breaking change for users that rely the
scores
---------
Signed-off-by: BubbleCal <bubble-cal@outlook.com >
2024-08-08 15:33:15 +08:00
Lance Release
4db554eea5
Updating package-lock.json
2024-08-07 20:56:12 +00:00
Lance Release
101066788d
Bump version: 0.9.0-beta.0 → 0.9.0
2024-08-07 20:55:53 +00:00
Lance Release
c4135d9d30
Bump version: 0.8.0 → 0.9.0-beta.0
2024-08-07 20:55:52 +00:00
Lance Release
ec39d98571
Bump version: 0.12.0-beta.0 → 0.12.0
2024-08-07 20:55:40 +00:00
Lance Release
0cb37f0e5e
Bump version: 0.11.0 → 0.12.0-beta.0
2024-08-07 20:55:39 +00:00
Gagan Bhullar
24e3507ee2
fix(node): export optimize options ( #1518 )
...
PR fixes #1514
2024-08-07 13:15:51 -07:00
Lei Xu
2bdf0a02f9
feat!: upgrade lance to 0.16 ( #1519 )
2024-08-07 13:15:22 -07:00
Gagan Bhullar
32123713fd
feat(python): optimize stats repr method ( #1510 )
...
PR fixes #1507
2024-08-07 08:47:52 -07:00
Gagan Bhullar
d5a01ffe7b
feat(python): index config repr method ( #1509 )
...
PR fixes #1506
2024-08-07 08:46:46 -07:00
Ayush Chaurasia
e01045692c
feat(python): support embedding functions in remote table ( #1405 )
2024-08-07 20:22:43 +05:30
