rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2025-12-26 14:49:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
370 Commits 117 Branches 395 Tags
2f2cb984d489e22c87603e345c4e108a1d350a15
Commit Graph

109 Commits

Author SHA1 Message Date
Rob Meng
2f2cb984d4 [breaking change] make schema a property (#414) 2023-08-11 18:58:41 -04:00
Lei Xu
aa91f35a28 [Python][Remote] Raise meaningful exception for to_arrow() / to_pandas() (#413) 2023-08-08 14:40:09 -07:00
Rob Meng
fd65887d87 implement remote drop table call (#411) 
Also moves `request_id` to header instead of request param
 2023-08-08 13:24:16 -04:00
Chang She
a54d1e5618 Automatically convert pydantic model (#400) 
Saves users from having to explicitly call
`LanceModel.to_arrow_schema()` when creating an empty table.
See new docs for full details.

---------

Co-authored-by: Chang She <chang@lancedb.com>
 2023-08-06 14:50:03 -07:00
Ayush Chaurasia
bbfadfe58d [python] Allow adding via iterators (#391) 
Makes the following work so all the formats accepted by `create_table()`
are also accepted by `add()`
```
import lancedb
import pyarrow as pa

db = lancedb.connect("/tmp")

def make_batches():
    for i in range(5):
        yield pa.RecordBatch.from_arrays(
            [
                pa.array([[3.1, 4.1], [5.9, 26.5]]),
                pa.array(["foo", "bar"]),
                pa.array([10.0, 20.0]),
            ],
            ["vector", "item", "price"],
        )

schema = pa.schema([
    pa.field("vector", pa.list_(pa.float32())),
    pa.field("item", pa.utf8()),
    pa.field("price", pa.float32()),
])

tbl = db.create_table("table4", make_batches(), schema=schema)
tbl.add(make_batches())
```
 2023-08-04 12:49:44 -07:00
Lei Xu
f0e1290ae6 Restrict semver version to 3.0 (#389) 2023-07-31 22:26:24 -07:00
Chang She
4b45128bd6 add LanceModel to docs (#386) 
Co-authored-by: Chang She <chang@lancedb.com>
 2023-07-31 15:12:02 -04:00
Lance Release
b06e214d29 [python] Bump version: 0.1.15 → 0.1.16 2023-07-31 18:32:40 +00:00
Chang She
c1f8feb6ed make pandas an optional dependency in lancedb as well (#385) 2023-07-31 14:08:58 -04:00
Chang She
cada35d5b7 Improve pydantic integration (#384) 2023-07-31 12:16:44 -04:00
Chang She
2d25c263e9 Implement drop table if exists (#383) 2023-07-31 10:25:09 +02:00
gsilvestrin
1daecac648 fix(python): Pin pylance and add pandas as test dependency (#373) 2023-07-27 15:21:45 -07:00
Lance Release
0e4050e706 [python] Bump version: 0.1.14 → 0.1.15 2023-07-25 18:58:44 +00:00
Rob Meng
147796ffcd bump lance version for vectordb, fix minor bugs in lancedb remote client (#365) 2023-07-24 21:30:57 -04:00
Rob Meng
8325979bb8 dont print apikey in remote client toString, add hostoverride to python client (#353) 2023-07-23 18:44:00 -04:00
Lance Release
1cbfc1bbf4 [python] Bump version: 0.1.13 → 0.1.14 2023-07-20 20:06:15 +00:00
Will Jones
0cf40c8da3 fix: only use util function to build filesystem (#339) 2023-07-20 10:41:50 -07:00
Rob Meng
8233c689c3 fix remote SDK (#342) 2023-07-20 02:01:13 -04:00
Lance Release
f4ce86e12c [python] Bump version: 0.1.12 → 0.1.13 2023-07-19 03:09:50 +00:00
Lei Xu
63acdc2069 [Python] Support pydantic v1 as well (#337) 
Support both Pydantic v1 and v2 (breaking changes)
 2023-07-18 19:53:09 -07:00
Lance Release
5e3167da83 [python] Bump version: 0.1.11 → 0.1.12 2023-07-19 01:18:28 +00:00
Lei Xu
f09db4a6d6 [Python] Do not return Table count for every add operation (#328) 
`Table::count()` will be linearly slower with more fragments ingested.
 2023-07-18 17:11:17 -07:00
Will Jones
fb97b03a51 feat: pass AWS_ENDPOINT environment variable down (#330) 
Tested locally against minio.
 2023-07-18 15:07:26 -07:00
Lei Xu
141b6647a8 [Python] Fix bumpversion.cfg (#327) 2023-07-18 09:18:14 -07:00
Lei Xu
a86bc05131 [Bug] Fix dataset path check in Table::open (#326) 
Fixed a bug that prevents to open remote tables.
 2023-07-18 08:45:10 -07:00
Will Jones
3537afb2c3 docs: show how to delete rows in user guide (#309) 
Closes #265
 2023-07-18 08:19:48 -07:00
Lei Xu
23f5dddc7c [Rust] Checkout a version of dataset. (#321) 
* `Table::open()` from absolute path, and gives the responsibility of
organizing metadata out of Table object
* Fix Clippy warnings
* Add `Table::checkout(version)` API
 2023-07-17 17:29:58 -07:00
Lei Xu
030f07e7f0 Bump minimal lance version to 0.5.8 (#318) 2023-07-17 12:41:29 -07:00
Lei Xu
088e745e1d [Python] Create table with Iterator[RecordBatch] and add docs (#316) 2023-07-16 21:45:55 -07:00
Lei Xu
7a57cddb2c [Python] Add records to remote (#315) 2023-07-16 13:24:38 -07:00
Lei Xu
8ff5f88916 [Python] Bug fixes in remote API (#314) 2023-07-16 11:09:19 -07:00
Lei Xu
028a6e433d [Python] Get table schema (#313) 2023-07-15 17:39:37 -07:00
Lei Xu
c62e4ca1eb Bump lance version to 0.5.7 (#311) 2023-07-14 17:17:31 -07:00
Chang She
2fdcb307eb [python] Fix a few minor bugs (#304) 2023-07-15 03:47:42 +08:00
Lei Xu
8c9edafccc [Doc] Add more Python integrations documents (#299) 2023-07-13 17:09:39 -07:00
Lei Xu
08944bf4fd [Python] Convert Pydantic Model to Arrow Schema (#291) 
Provide utility to automatically convert Pydantic model to Arrow Schema

Closes #256
 2023-07-13 11:16:37 -07:00
Lei Xu
0e7ae5dfbf [Python] Fix list type conversion to JSON and temporal types (#274) 2023-07-11 11:05:51 -07:00
Lei Xu
9f603f73a9 [Python] Schema to JSON (#272) 2023-07-10 18:11:24 -07:00
Lei Xu
9ef846929b [Python] List tables from remote service (#262) 2023-07-09 23:58:03 -07:00
Lei Xu
97364a2514 Bump to v0.1.10-python 2023-07-09 21:52:11 -07:00
Lei Xu
e6c6da6104 [Python] Initial support of cloud API (#260) 
Support connect with remote database, and implement Search API
 2023-07-07 15:41:15 -07:00
Chang She
e2325c634b Allow creation of an empty table (#254) 
It's inconvenient to always require data at table creation time.
Here we enable you to create an empty table and add data and set schema
later.

---------

Co-authored-by: Chang She <chang@lancedb.com>
 2023-07-06 20:44:58 -07:00
Chang She
507eeae9c8 Set default to error instead of drop (#259) 
when encountering bad input data, we can default to principle of least
surprise and raise an exception.

Co-authored-by: Chang She <chang@lancedb.com>
 2023-07-05 22:44:18 -07:00
Chang She
3c46d7f268 Handle NaN input data (#241) 
Sometimes LangChain would insert a single `[np.nan]` as a placeholder if
the embedding function failed. This causes a problem for Lance format
because then the array can't be stored as a FixedSizedListArray.

Instead:
1. By default we remove rows with embedding lengths less than the
maximum length in the batch
2. If `strict=True` kwargs is set to True, then a `ValueError` is raised
if the embeddings aren't all the same length

---------

Co-authored-by: Chang She <chang@lancedb.com>
 2023-07-04 20:00:46 -07:00
Lei Xu
fc725c99f0 [Node] Create Table with WriteMode (#246) 
Support `createTable(name, data, mode?)`  to be consistent with Python.

Closes #242
 2023-07-03 17:04:21 -07:00
Leon Yee
eb5bcda337 Error implementations (#232) 
Solves #216 by adding a check on table open for existence of the
`.lance` file. Does not check for it for remote connections.
 2023-06-27 16:48:31 -07:00
Lei Xu
4bc676e26a [Python] Support replace during create_index (#233) 
Closes #214
 2023-06-27 16:02:07 -07:00
Philip Kung
313e66c4c5 Specify and Index Column for Vector Search (#217) 2023-06-26 16:11:08 -07:00
Will Jones
0e4c52b8a6 bump python module version 2023-06-26 11:25:39 -07:00
Will Jones
ad48242ffb feat: support for deletion (#219) 
Also upgrades Arrow and Lance.
 2023-06-23 18:09:07 -07:00