rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-10-14 07:42:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,726 Commits 374 Branches 556 Tags
docs/python-api-reference-gaps
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Lei Xu 357197bacc chore!: change support python version from 3.10 to 3.13 (#2955) 
Python 3.9 is EOL since Oct 2025. and last two pyarrow builts were
against python3.10-3.13.

* This PR is contributed by codex-gpt5.2
 2026-01-30 01:47:50 +08:00
Lei Xu ad51e2dd1f fix: support pydantic list of structs or optional struct (#2953) 
Closes #2950

*This code is generated by codex-gpt5.2*
 2026-01-28 21:08:18 -08:00
Lei Xu a67a7b4b42 chore: use stable lance (#2398) 
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Chores**
- Updated workspace dependencies to use a stable release version for
improved consistency and reliability. No changes to application features
or functionality.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
 2025-05-21 22:34:29 -07:00
Lei Xu 496846e532 chore: bump lance version (#2397) 
- Bump lance version and prepare a new release.
- Bump rust toolchain to 1.86, because GHA ubuntu does not have 1.83
`cargo-fmt` anymore
 2025-05-21 14:15:55 -07:00
Lei Xu 4708b60bb1 chore: cargo update on main (#2331) 
Fix test failures on main
 2025-04-12 09:00:47 -05:00
Lei Xu 080ea2f9a4 chore: fix 1.86 warnings (#2312) 
Fix rust 1.86 warnings
 2025-04-12 08:29:10 -05:00
Lei Xu a38f784081 chore: add numpy as dependency (#2308) 2025-04-04 10:33:39 -07:00
Lei Xu f52d05d3fa feat: add columns using pyarrow schema (#2284) 2025-03-28 08:51:50 -07:00
Lei Xu 7c12d497b0 ci: bump python to 3.12 in GHA (#2169) 2025-03-01 17:24:02 -08:00
Lei Xu 6fa1f37506 docs: improve pydantic integration docs (#2136) 
Address usage mistakes in
https://github.com/lancedb/lancedb/issues/2135.

* Add example of how to use `LanceModel` and `Vector` decorator
* Add test for pydantic doc
* Fix the example to directly use LanceModel instead of calling
`MyModel.to_arrow_schema()` in the example.
* Add cross-reference link to pydantic doc site
* Configure mkdocs to watch code changes in python directory.
 2025-02-21 12:48:37 -08:00
Lei Xu 1865f7decf fix: support optional nested pydantic model (#2130) 
Closes #2129
 2025-02-17 20:43:13 -08:00
Lei Xu f76c4a5ce1 chore: add pyright static type checking and fix some of the table interface (#1996) 
* Enable `pyright` in the project
* Fixed some pyright typing errors in `table.py`
 2025-01-04 15:24:58 -08:00
Lei Xu 397813f6a4 chore: bump pylance to 0.21.1b1 (#1989) 2024-12-31 15:34:27 -08:00
Lei Xu 50c30c5d34 chore(python): fix typo of the synchronized checkout API (#1988) 2024-12-30 18:54:31 -08:00
Lei Xu 347515aa51 fix: support list of numpy f16 floats as query vector (#1931) 
User reported on Discord, when using
`table.vector_search([np.float16(1.0), np.float16(2.0), ...])`, it
yields `TypeError: 'numpy.float16' object is not iterable`
 2024-12-10 16:17:28 -08:00
Lei Xu c78a9849b4 ci: upgrade version of upload-pages-artifact and deploy-pages (#1917) 
For
https://github.blog/changelog/2024-12-05-deprecation-notice-github-pages-actions-to-require-artifacts-actions-v4-on-github-com/
 2024-12-06 10:45:24 -05:00
Lei Xu bd82e1f66d feat(python): add support for Azure OpenAPI SDK (#1906) 
Closes #1699
 2024-12-04 13:09:38 -08:00
Lei Xu 2ded17452b fix(python)!: handle bad openai embeddings gracefully (#1873) 
BREAKING-CHANGE: change Pydantic Vector field to be nullable by default.
Closes #1577
 2024-11-23 13:33:52 -08:00
Lei Xu d369233b3d feat: bump lance to 0.20.0b2 (#1865) 
Bump lance version.
Upstream change log:
https://github.com/lancedb/lance/releases/tag/v0.20.0-beta.2
 2024-11-21 13:16:59 -08:00
Lei Xu 267aa83bf8 feat(python): check vector query is not None (#1847) 
Fix the type hints of `nearest_to` method, and raise `ValueError` when
the input is None
 2024-11-18 14:15:22 -08:00
Lei Xu 4c9bab0d92 fix: use pandas with pydantic embedding column (#1818) 
* Make Pandas `DataFrame` works with embedding function + Subset of
columns
* Make `lancedb.create_table()` work with embedding function
 2024-11-11 14:48:56 -08:00
Lei Xu f0e7f5f665 ci: change to use github runner (#1708) 
Use github runner
 2024-09-27 17:53:05 -07:00
Lei Xu 7c314d61cc chore: add error handling for openai embedding generation (#1680) 2024-09-23 12:10:56 +05:30
Lei Xu 7a8d2f37c4 feat(rust): add with_row_id to rust SDK (#1683) 2024-09-21 21:26:19 -07:00
Lei Xu 915d828cee feat!: set embeddings to Null if embedding function return invalid results (#1674) 2024-09-19 23:16:20 -07:00
Lei Xu 32af962c0c feat: fix creating empty table and creating table by a list of RecordBatch for remote python sdk (#1650) 
Closes #1637
 2024-09-14 11:33:34 -07:00
Lei Xu c02ee3c80c chore: make remote client a context manager (#1648) 
Allow `RemoteLanceDBClient` to be used as context manager
 2024-09-13 22:08:48 -07:00
Lei XuandLu Qiu 4ee7225e91 ci: public java package (#1485) 
Co-authored-by: Lu Qiu <luqiujob@gmail.com>
 2024-09-05 11:48:48 -07:00
Lei Xu 5857cb4c6e docs: add a section to describe scalar index (#1495) 2024-08-16 18:48:29 -07:00
Lei Xu 694ca30c7c feat(nodejs): add bitmap and label list index types in nodejs (#1532) 2024-08-11 12:06:02 -07:00
Lei Xu b2317c904d feat: create bitmap and label list scalar index using python async api (#1529) 
* Expose `bitmap` and `LabelList` scalar index type via Rust and Async
Python API
* Add documents
 2024-08-11 09:16:11 -07:00
Lei Xu 2bdf0a02f9 feat!: upgrade lance to 0.16 (#1519) 2024-08-07 13:15:22 -07:00
Lei Xu c9c61eb060 docs: expose merge_insert doc for remote python SDK (#1464) 
`merge_insert` API is not shown up on
[`RemoteTable`](https://lancedb.github.io/lancedb/python/saas-python/#lancedb.remote.table.RemoteTable)
today

* Also bump `ruff` version as well
 2024-07-22 10:48:16 -07:00
Lei Xu 0708428357 feat: support update over binary field (#1440) 2024-07-12 09:22:00 -07:00
Lei Xu 58c2cd01a5 docs: add fast search to openapi.yml (#1435) 2024-07-08 11:55:45 -07:00
Lei Xu 3c6c21c137 feat(rust): enable fast search flag in rust (#1432) 2024-07-07 09:46:41 -07:00
Lei Xu fd5ca20f34 chore: bump lance to 0.14 (#1430) 2024-07-06 14:10:42 -07:00
Lei Xu ef30f87fd1 chore: propagate error for table index stats (#1426) 2024-07-04 14:53:49 -07:00
Lei XuandWeston Pace 020a437230 docs: add merge insert, create index and create scalar index to public rest api doc (#1420) 
Added 3 APIs doc publicly:
- `merge_insert`
- `create_index`
- `create_scalar_index`

---------

Co-authored-by: Weston Pace <weston.pace@gmail.com>
 2024-07-01 12:52:27 -07:00
Lei Xu e780b2f51c ci: fix nodejs doc test (#1419) 
Fixed nodejs doctest failures due to compiling JNI node.
 2024-07-01 10:21:41 -07:00
Lei Xu d6485f1215 docs: add openapi rest api page (#1413) 2024-06-27 21:32:34 -07:00
Lei Xu 56b4fd2bd9 feat(rust): allow to create execution plan on queries (#1350) 2024-05-31 17:33:58 -07:00
Lei Xu b272408b05 chore: fix main branch test failure (#1240) 2024-04-24 13:49:37 -07:00
Lei Xu e6ff3d848b chore: bump to 0.10.8 (#1187) 2024-04-05 16:34:50 -07:00
Lei Xu 1d3325dcc5 chore: bump lance version (#1185) 
Bump lance version to `0.10.7`
 2024-04-05 16:34:50 -07:00
Lei Xu 7e5a54b76a chore: add social link footer (#1177) 2024-04-05 16:34:50 -07:00
Lei Xu 3f14938392 chore: pass str instead of String to build table names (#1178) 2024-04-05 16:34:50 -07:00
Lei Xu 473ef7e426 chore: validate table name (#1146) 
Closes #1129
 2024-04-05 16:33:37 -07:00
Lei Xu 1aaaeff511 chore: bump lance to 0.10.5 (#1145) 2024-04-05 16:33:37 -07:00
Lei Xu f27167017b chore: bump lance to 0.10.2 (#1061) 2024-04-05 16:31:45 -07:00
Lei Xu 80e4d14c02 chore: bump pylance to 0.9.18 (#1011) 2024-04-05 16:30:40 -07:00
Lei Xu 84edf56995 chore: add global cargo config to enable minimal cpu target (#925) 
* Closes #895 
* Fix cargo clippy
 2024-04-05 16:29:13 -07:00
Lei Xu bd2d187538 ci: bump to new version of python action to use node 20 gIthub action runtime (#909) 
Github action is deprecating old node-16 runtime.
 2024-04-05 16:29:05 -07:00
Lei Xu 8e139012e2 fix(node): pass AWS credentials to db level operations (#908) 
Passed the following tests

```ts
const keyId = process.env.AWS_ACCESS_KEY_ID;
const secretKey = process.env.AWS_SECRET_ACCESS_KEY;
const sessionToken = process.env.AWS_SESSION_TOKEN;
const region = process.env.AWS_REGION;

const db = await lancedb.connect({
  uri: "s3://bucket/path",
  awsCredentials: {
    accessKeyId: keyId,
    secretKey: secretKey,
    sessionToken: sessionToken,
  },
  awsRegion: region,
} as lancedb.ConnectionOptions);

  console.log(await db.createTable("test", [{ vector: [1, 2, 3] }]));
  console.log(await db.tableNames());
  console.log(await db.dropTable("test"))
```
 2024-04-05 16:28:56 -07:00
Lei Xu a617ad35ff ci: change apple silicon runner to free OSS macos-14 target (#901) 2024-04-05 16:28:56 -07:00
Lei Xu 911d063237 doc: fix js example of create index (#886) 2024-04-05 16:28:56 -07:00
Lei Xu 12e776821a doc: use snippet for rust code example and make sure rust examples run through CI (#885) 2024-04-05 16:28:56 -07:00
Lei Xu c6e5eb0398 fix: fix doc build to include the source snippet correctly (#883) 2024-04-05 16:28:56 -07:00
Lei Xu e7fdb931de chore: convert all js doc test to use snippet. (#881) 2024-04-05 16:28:56 -07:00
Lei Xu d811b89de2 doc: use code snippet for typescript examples (#880) 
The typescript code is in a fully function file, that will be run via the CI.
 2024-04-05 16:28:56 -07:00
Lei Xu f2e29eb004 chore: upgrade lance, pylance and datafusion (#879) 2024-04-05 16:28:56 -07:00
Lei XuandChang She 36dbf47d60 chore: add one rust SDK e2e example (#876) 
Co-authored-by: Chang She <759245+changhiskhan@users.noreply.github.com>
 2024-04-05 16:28:56 -07:00
Lei Xu fd2fd94862 doc: update quick start for full rust example (#872) 2024-04-05 16:28:56 -07:00
Lei Xu faa5912c3f chore: bump github actions to v4 due to GHA warnings of node version deprecation (#874) 2024-04-05 16:28:56 -07:00
Lei Xu 7274c913a8 feat(rust): provide connect and connect_with_options in Rust SDK (#871) 
* Bring the feature parity of Rust connect methods.
* A global connect method that can connect to local and remote / cloud
table, as the same as in js/python today.
 2024-04-05 16:28:18 -07:00
Lei Xu a192c1a9b1 chore(rust): simplified version of optimize (#869) 
Consolidate various optimize() into one method, similar to postgres
VACCUM in the process of preparing Rust API for public use
 2024-04-05 16:28:18 -07:00
Lei Xu cef0293985 feat(napi): Issue queries as node SDK (#868) 
* Query as a fluent API and `AsyncIterator<RecordBatch>`
* Much more docs
* Add tests for auto infer vector search columns with different
dimensions.
 2024-04-05 16:28:18 -07:00
Lei Xu 0664eee38d fix: release build for node sdk (#861) 2024-04-05 16:27:59 -07:00
Lei Xu 8b04d8fef6 feat: improve the rust table query API and documents (#860) 
* Easy to type
* Handle `String, &str, [String] and [&str]` well without manual
conversion
* Fix function name to be verb
* Improve docstring of Rust.
* Promote `query` and `search()` to public `Table` trait
 2024-04-05 16:27:51 -07:00
Lei Xu 68e2bb0b2d doc: update rust readme to include crate and docs.rs links (#859) 2024-04-05 16:27:51 -07:00
Lei Xu db4a979278 feat(napi): Provide a new createIndex API in the napi SDK. (#857) 2024-04-05 16:27:51 -07:00
Lei Xu dfabbe9081 feat(rust): create index API improvement (#853) 
* Extract a minimal Table interface in Rust SDK
* Make create_index composable in Rust.
* Fix compiling issues from ffi
 2024-04-05 16:27:51 -07:00
Lei Xu efcaa433fe feat: rework NodeJS SDK using napi (#847) 
Use Napi to write a Node.js SDK that follows Polars for better
maintainability, while keeping most of the logic in Rust.
 2024-04-05 16:27:51 -07:00
Lei Xu 65c1d8bc4c feat: change create table to accept Arrow table (#845) 2024-04-05 16:27:50 -07:00
Lei Xu a78e07907c chore(rust): provide a Connection trait to match python and nodejs SDK (#846) 
In NodeJS and Python, LanceDB establishes a connection to a db. In Rust
core, it is called Database.
We should be consistent with the naming.
 2024-04-05 16:27:42 -07:00
Lei Xu d8befeeea2 feat(js): add helper function to create Arrow Table with schema (#838) 
Support to make Apache Arrow Table from an array of javascript Records,
with optionally provided Schema.
 2024-04-05 16:27:42 -07:00
Lei Xu 49de13c65a doc: add index page for rust crate (#839) 
Rust API doc for the braves
 2024-04-05 16:27:42 -07:00
Lei Xu 97d033dfd6 bug: add a test for fp16 (#837) 
Add test to ingest fp16 to a database
 2024-04-05 16:27:42 -07:00
Lei Xu 5b2c602fb3 doc: improve docs for nodejs connect functions (#833) 
* improve the docstring for NodeJS connect functions and
`ConnectOptions` parameters.
* Simplify `npm run build` steps.
 2024-04-05 16:27:32 -07:00
Lei Xu 6f7cb75b07 chore: remove black as dependency (#808) 
We use `ruff` in CI and dev workflow now.
 2024-04-05 16:25:14 -07:00
Lei Xu 073a2a1b28 chore: bump lance to 0.9.5 (#790) 2024-04-05 16:25:14 -07:00
Lei Xu 45b006d68c chore: remove black as dependency (#808) 
We use `ruff` in CI and dev workflow now.
 2024-04-05 16:25:02 -07:00
Lei Xu 138760bc6e chore: bump pylance to 0.9.2 (#754) 2024-04-05 16:25:02 -07:00
Lei Xu 4c8690549a chore: bump lance to 0.9.5 (#790) 2024-04-05 16:25:01 -07:00
Lei Xu 56db257ea9 chore: bump pylance to 0.9.2 (#754) 2024-04-05 16:24:47 -07:00
Lei Xu 1d79e9168e chore: bump lance version to 0.9 (#715) 2024-04-05 16:24:30 -07:00
Lei Xu 49306a99ba chore: apple silicon runner (#633) 
Close #632
 2024-04-05 16:23:49 -07:00
Lei Xu 86efd36689 chore: improve create_table API consistency between local and remote SDK (#627) 2024-04-05 16:23:47 -07:00
Lei Xu 3855bdf986 chore: bump lance to 8.10 (#622) 2024-04-05 16:23:14 -07:00
Lei Xu b5e57ebce3 doc: add doc to use GPU for indexing (#611) 2024-04-05 16:22:59 -07:00
Lei Xu 8dc10180b0 feat(python,js): deletion operation on remote tables (#568) 2024-04-05 16:22:59 -07:00
Lei Xu 7b5bfadab2 chore: bump lance to 0.8.5 (#561) 
Bump lance to 0.5.8
 2024-04-05 16:22:59 -07:00
Lei XuandRob Meng a26c8f3316 feat: use GPU for index creation. (#540) 
Bump lance to 0.8.3 to include GPU training

---------

Co-authored-by: Rob Meng <rob.xu.meng@gmail.com>
 2023-10-05 20:49:00 -07:00
Lei Xu 8b815ef5a8 chore: upgrade lance to 0.8.1 (#536) 
Bump to lance 0.8.1 for both javascript and python sdk
 2023-10-03 14:29:18 -07:00
Lei Xu b315ea3978 [Python] Pydantic vector field with default value (#474) 
Rename `lance.pydantic.vector` to `Vector` and deprecate `vector(dim)`
 2023-09-08 22:35:31 -07:00
Lei Xu 6799613109 feat: upgrade lance to 0.7.3 (#481) 2023-09-07 17:01:45 -07:00
Lei Xu 0f26915d22 [Rust] schema coerce and vector column inference (#476) 
Split the rust core from #466 for easy review and less merge conflicts.
 2023-09-06 10:00:46 -07:00
Lei Xu 9921b2a4e5 [Node] Use index by default (#422) 2023-08-11 15:26:44 -07:00
Lei Xu aa91f35a28 [Python][Remote] Raise meaningful exception for to_arrow() / to_pandas() (#413) 2023-08-08 14:40:09 -07:00
Lei Xu f0e1290ae6 Restrict semver version to 3.0 (#389) 2023-07-31 22:26:24 -07:00