e0277383a5
feat(python): add optional threadpool for batch requests ( #981 )
...
Currently if a batch request is given to the remote API, each query is
sent sequentially. We should allow the user to specify a threadpool.
2024-02-16 20:22:22 -08:00
1b990983b3
Qian/make vector col optional ( #950 )
...
remote SDK tests were completed through lancedb_integtest
2024-02-12 16:35:44 -08:00
a9727eb318
feat: add support for filter during merge insert when matched ( #948 )
...
Closes #940
2024-02-09 10:26:14 -08:00
48d55bf952
added error msg to SaaS APIs ( #852 )
...
1. improved error msg for SaaS create_table and create_index
---------
Co-authored-by: Chang She <759245+changhiskhan@users.noreply.github.com >
2024-02-09 10:07:47 -08:00
d2e71c8b08
feat: add a filterable count_rows to all the lancedb APIs ( #913 )
...
A `count_rows` method that takes a filter was recently added to
`LanceTable`. This PR adds it everywhere else except `RemoteTable` (that
will come soon).
2024-02-08 09:40:29 -08:00
7f8637a0b4
feat: add merge_insert to the node and rust APIs ( #915 )
2024-02-02 13:16:51 -08:00
a248d7feec
fix: add request retry to python client ( #917 )
...
Adds capability to the remote python SDK to retry requests (fixes #911 )
This can be configured through environment:
- `LANCE_CLIENT_MAX_RETRIES`= total number of retries. Set to 0 to
disable retries. default = 3
- `LANCE_CLIENT_CONNECT_RETRIES` = number of times to retry request in
case of TCP connect failure. default = 3
- `LANCE_CLIENT_READ_RETRIES` = number of times to retry request in case
of HTTP request failure. default = 3
- `LANCE_CLIENT_RETRY_STATUSES` = http statuses for which the request
will be retried. passed as comma separated list of ints. default `500,
502, 503`
- `LANCE_CLIENT_RETRY_BACKOFF_FACTOR` = controls time between retry
requests. see
[here](https://github.com/urllib3/urllib3/blob/23f2287eb526d9384dddeedb6f6345e263bb9b86/src/urllib3/util/retry.py#L141-L146 ).
default = 0.25
Only read requests will be retried:
- list table names
- query
- describe table
- list table indices
This does not add retry capabilities for writes as it could possibly
cause issues in the case where the retried write isn't idempotent. For
example, in the case where the LB times-out the request but the server
completes the request anyway, we might not want to blindly retry an
insert request.
2024-02-02 11:27:29 -05:00
cc9473a94a
docs: add cleanup_old_versions and compact_files to Table for documentation purposes ( #900 )
...
Closes #819
2024-02-01 15:06:00 -08:00
d77e95a4f4
feat: upgrade to lance 0.9.11 and expose merge_insert ( #906 )
...
This adds the python bindings requested in #870 The javascript/rust
bindings will be added in a future PR.
2024-02-01 11:36:29 -08:00
5c5e23bbb9
chore(python): Temporarily extend remote connection timeout ( #888 )
...
Context - https://etoai.slack.com/archives/C05NC5YSW5V/p1706371205883149
2024-01-29 17:34:33 +05:30
b4d451ed21
extend timeout for requests.get and requests.post ( #848 )
2024-01-22 20:31:39 -08:00
83ed8d1e49
bug: add a test for fp16 ( #837 )
...
Add test to ingest fp16 to a database
2024-01-20 16:23:28 -08:00
c89d5e6e6d
fix: remote python client closes idle connections ( #831 )
2024-01-19 17:28:36 -05:00
d012db24c2
ci: lint and enforce linting ( #829 )
...
@eddyxu added instructions for linting here:
https://github.com/lancedb/lancedb/blob/7af213801a091cd5afb0f0814e184fc0b852de47/python/README.md?plain=1#L45-L50
However, we had a lot of failures and weren't checking this in CI. This
PR fixes all lints and adds a check to CI to keep us in compliance with
the lints.
2024-01-19 13:09:14 -08:00
99adfe065a
use requests instead of aiohttp for underlying http client ( #803 )
...
instead of starting and stopping the current thread's event loop on
every http call, just make an http call.
2024-01-10 00:07:50 -05:00
7773bda7ee
feat(python): first cut batch queries for remote api ( #753 )
...
issue separate requests under the hood and concatenate results
2023-12-29 15:33:03 -08:00
dc5126d8d1
feat: add the ability to create scalar indices ( #679 )
...
This is a pretty direct binding to the underlying lance capability
2023-12-21 09:50:10 -08:00
57207eff4a
implement update for remote clients ( #706 )
2023-12-15 09:06:40 -05:00
2d78bff120
feat: pass vector column name to remote backend ( #710 )
...
pass vector column name to remote as well.
`vector_column` is already part of `Query` just declearing it as part to
`remote.VectorQuery` as well
2023-12-15 00:19:08 -05:00
f6bbe199dc
Qian/minor fix doc ( #695 )
2023-12-08 09:58:53 -08:00
aca785ff98
saas python sdk doc ( #692 )
...
<img width="256" alt="Screenshot 2023-12-07 at 11 55 41 AM"
src="https://github.com/lancedb/lancedb/assets/1305083/259bf234-9b3b-4c5d-af45-c7f3fada2cc7 ">
2023-12-07 14:47:56 -08:00
6eb662de9b
fix: python remote correct open_table error message ( #659 )
2023-11-24 19:28:33 -05:00
d8e3e54226
feat(python): expose index cache size ( #655 )
...
This is to enable https://github.com/lancedb/lancedb/issues/641 .
Should be merged after https://github.com/lancedb/lance/pull/1587 is
released.
2023-11-18 14:17:40 -08:00
1cf8a3e4e0
SaaS create_index API ( #649 )
2023-11-15 19:12:52 -05:00
1c872ce501
feat: add RemoteTable.version in Python ( #644 )
...
Please note: this is not tested as we don't have a server here and
testing against a mock object wouldn't be that interesting.
2023-11-13 21:43:48 +01:00
662968559d
fix saas open_table and table_names issues ( #640 )
...
- added check whether a table exists in SaaS open_table
- remove prefilter not supported warning in SaaS search
- fixed issues for SaaS table_names
2023-11-07 17:34:38 -08:00
554e068917
chore: improve create_table API consistency between local and remote SDK ( #627 )
2023-11-03 13:15:11 -07:00
7eec2b8f9a
Qian/query option doc ( #615 )
...
- API documentation improvement for queries (table.search)
- a small bug fix for the remote API on create_table
2023-10-31 19:50:05 -07:00
d46bc5dd6e
list table pagination draft ( #574 )
2023-10-16 21:09:20 -07:00
fe64fc4671
feat(python,js): deletion operation on remote tables ( #568 )
2023-10-14 15:47:19 -07:00
a26c8f3316
feat: use GPU for index creation. ( #540 )
...
Bump lance to 0.8.3 to include GPU training
---------
Co-authored-by: Rob Meng <rob.xu.meng@gmail.com >
2023-10-05 20:49:00 -07:00
693bca1eba
feat(python): expose prefilter to lancedb ( #522 )
...
We have experimental support for prefiltering (without ANN) in pylance.
This means that we can now apply a filter BEFORE vector search is
performed. This can be done via the `.where(filter_string,
prefilter=True)` kwargs of the query.
Limitations:
- When connecting to LanceDB cloud, `prefilter=True` will raise
NotImplemented
- When an ANN index is present, `prefilter=True` will raise
NotImplemented
- This option is not available for full text search query
- This option is not available for empty search query (just
filter/project)
Additional changes in this PR:
- Bump pylance version to v0.8.0 which supports the experimental
prefiltering.
---------
Co-authored-by: Chang She <chang@lancedb.com >
2023-10-01 10:34:12 -07:00
a695fb8030
fix import attr to use import attrs ( #510 )
...
Thanks to #508 , I used `attr` instead of the correct package `attrs`
s/attr/attrs
2023-09-23 00:30:56 -04:00
31dad71c94
multi-modal embedding-function ( #484 )
2023-09-16 21:23:51 -04:00
9a9a73a65d
[python] Use pydantic for embedding function persistence ( #467 )
...
1. Support persistent embedding function so users can just search using
query string
2. Add fixed size list conversion for multiple vector columns
3. Add support for empty query (just apply select/where/limit).
4. Refactor and simplify some of the data prep code
---------
Co-authored-by: Chang She <chang@lancedb.com >
Co-authored-by: Weston Pace <weston.pace@gmail.com >
2023-09-05 21:30:45 -07:00
aa91f35a28
[Python][Remote] Raise meaningful exception for to_arrow() / to_pandas() ( #413 )
2023-08-08 14:40:09 -07:00
fd65887d87
implement remote drop table call ( #411 )
...
Also moves `request_id` to header instead of request param
2023-08-08 13:24:16 -04:00
c1f8feb6ed
make pandas an optional dependency in lancedb as well ( #385 )
2023-07-31 14:08:58 -04:00
147796ffcd
bump lance version for vectordb, fix minor bugs in lancedb remote client ( #365 )
2023-07-24 21:30:57 -04:00
8325979bb8
dont print apikey in remote client toString, add hostoverride to python client ( #353 )
2023-07-23 18:44:00 -04:00
8233c689c3
fix remote SDK ( #342 )
2023-07-20 02:01:13 -04:00
f09db4a6d6
[Python] Do not return Table count for every add operation ( #328 )
...
`Table::count()` will be linearly slower with more fragments ingested.
2023-07-18 17:11:17 -07:00
3537afb2c3
docs: show how to delete rows in user guide ( #309 )
...
Closes #265
2023-07-18 08:19:48 -07:00
7a57cddb2c
[Python] Add records to remote ( #315 )
2023-07-16 13:24:38 -07:00
8ff5f88916
[Python] Bug fixes in remote API ( #314 )
2023-07-16 11:09:19 -07:00
028a6e433d
[Python] Get table schema ( #313 )
2023-07-15 17:39:37 -07:00
9ef846929b
[Python] List tables from remote service ( #262 )
2023-07-09 23:58:03 -07:00
e6c6da6104
[Python] Initial support of cloud API ( #260 )
...
Support connect with remote database, and implement Search API
2023-07-07 15:41:15 -07:00
507eeae9c8
Set default to error instead of drop ( #259 )
...
when encountering bad input data, we can default to principle of least
surprise and raise an exception.
Co-authored-by: Chang She <chang@lancedb.com >
2023-07-05 22:44:18 -07:00
d1e8a97a2a
isort entire repo ( #200 )
2023-06-15 20:12:10 -04:00
Chang She
QianZhu
Weston Pace
Bert
Ayush Chaurasia
Lei Xu
Will Jones
Sebastian Law
Rob Meng
Rok Mihevc
Aidan