vinoyang
374fe0ad95
feat(rust): introduce Catalog trait and implement ListingCatalog ( #2148 )
...
Co-authored-by: Weston Pace <weston.pace@gmail.com >
2025-03-03 20:22:24 -08:00
Lei Xu
7c12d497b0
ci: bump python to 3.12 in GHA ( #2169 )
2025-03-01 17:24:02 -08:00
Weston Pace
d6b3ccb37b
feat: upgrade lance to 0.23.2 ( #2152 )
...
This also changes the pylance pin from `==0.23.2` to `~=0.23.2` which
should allow the pylance dependency to float a little. The pylance
dependency is actually not used for much anymore and so it should be
tolerant of patch changes.
2025-02-26 09:02:51 -08:00
Will Jones
e05c0cd87e
ci(node): check docs in CI ( #2084 )
...
* Make `npm run docs` fail if there are any warnings. This will catch
items missing from the API reference.
* Add a check in our CI to make sure `npm run dos` runs without warnings
and doesn't generate any new files (indicating it might be out-of-date.
* Hide constructors that aren't user facing.
* Remove unused enum `WriteMode`.
Closes #2068
2025-01-30 16:06:06 -08:00
Will Jones
a677a4b651
ci: fix arm64 windows cross compile build ( #2081 )
...
* Adds a CI job to check the cross compiled Windows ARM build.
* Didn't replace the test build because we need native build to run
tests. But for some reason (I forget why) we need cross compiled for
nodejs.
* Pinned crunchy to workaround
https://github.com/eira-fransham/crunchy/issues/13
This is needed to fix failure from
https://github.com/lancedb/lancedb/actions/runs/13020773184/job/36320719331
2025-01-30 09:24:20 -08:00
Will Jones
15f8f4d627
ci: check license headers ( #2076 )
...
Based on the same workflow in Lance.
2025-01-29 08:27:07 -08:00
Will Jones
6526d6c3b1
ci(rust): caching improvements (up to 2.8x faster builds) ( #2075 )
...
Some Rust jobs (such as
[Rust/linux](https://github.com/lancedb/lancedb/actions/runs/13019232960/job/36315830779 ))
take almost minutes. This can be a bit of a bottleneck.
* Two fixes to make caches more effective
* Check in `Cargo.lock` so that dependencies don't change much between
runs
* Added a new CI job to validate we can build without a lockfile
* Altered build commands so they don't have contradictory features and
therefore don't trigger multiple builds
Sadly, I don't think there's much to be done for windows-arm64, as much
of the compile time is because the base image is so bare we need to
install the build tools ourselves.
2025-01-29 08:26:45 -08:00
Will Jones
7920ecf66e
ci(python): stop using deprecated 2_24 manylinux for arm ( #2064 )
...
Based on changes made in Lance:
* https://github.com/lancedb/lance/pull/3409
* https://github.com/lancedb/lance/pull/3411
2025-01-23 15:00:34 -08:00
Mr. Doge
998c5f3f74
ci: add dbghelp.lib to sysroot-aarch64-pc-windows-msvc.sh ( #1975 ) ( #2008 )
...
successful runs:
https://github.com/FuPeiJiang/lancedb/actions/runs/12698662005
2025-01-09 14:24:09 -08:00
Will Jones
6eacae18c4
test: fix test failure from merge ( #2007 )
2025-01-09 11:27:24 -08:00
Will Jones
8b31540b21
ci: prevent stable release with preview lance ( #1995 )
...
Accidentally referenced a preview release in our stable release of
LanceDB. This adds a CI check to prevent that.
2025-01-06 08:54:14 -08:00
Lei Xu
f76c4a5ce1
chore: add pyright static type checking and fix some of the table interface ( #1996 )
...
* Enable `pyright` in the project
* Fixed some pyright typing errors in `table.py`
2025-01-04 15:24:58 -08:00
Will Jones
0a0f667bbd
chore: fix typos ( #1976 )
2024-12-24 12:50:54 -08:00
Will Jones
03753fd84b
ci(node): remove hardcoded toolchain from typescript release build ( #1974 )
...
We upgraded the toolchain in #1960 , but didn't realize we hardcoded it
in `npm-publish.yml`. I found if I just removed the hard-coded
toolchain, it selects the correct one.
This didn't fully fix Windows Arm, so I created a follow-up issue here:
https://github.com/lancedb/lancedb/issues/1975
2024-12-24 12:48:41 -08:00
Will Jones
27ef0bb0a2
ci(rust): check MSRV and upgrade toolchain ( #1960 )
...
* Upgrades our toolchain file to v1.83.0, since many dependencies now
have MSRV of 1.81.0
* Reverts Rust changes from #1946 that were working around this in a
dumb way
* Adding an MSRV check
* Reduce MSRV back to 1.78.0
2024-12-19 08:43:25 -08:00
Will Jones
25402ba6ec
chore: update lockfiles ( #1946 )
2024-12-18 08:43:33 -08:00
Will Jones
d11b2a6975
ci: fix python beta release to publish to fury ( #1937 )
...
We have been publishing all releases--even preview ones--to PyPI. This
was because of a faulty bash if statement. This PR fixes that
conditional.
2024-12-13 14:19:14 -08:00
Lei Xu
c78a9849b4
ci: upgrade version of upload-pages-artifact and deploy-pages ( #1917 )
...
For
https://github.blog/changelog/2024-12-05-deprecation-notice-github-pages-actions-to-require-artifacts-actions-v4-on-github-com/
2024-12-06 10:45:24 -05:00
Will Jones
8b628854d5
ci: fix nodejs release jobs ( #1912 )
...
* Clean up old commented out jobs
* Fix runner issue that caused these failures:
https://github.com/lancedb/lancedb/actions/runs/12186754094
2024-12-05 14:45:10 -08:00
Mr. Doge
c7d424b2f3
ci: aarch64-pc-windows-msvc ( #1890 )
...
`npm run pack-build -- -t $TARGET_TRIPLE`
was needed instead of
`npm run pack-build -t $TARGET_TRIPLE`
https://github.com/lancedb/lancedb/pull/1889
some documentation about `*-pc-windows-msvc` cross-compilation (from
alpine):
https://github.com/lancedb/lancedb/pull/1831#issuecomment-2497156918
only `arm64` in `matrix` config is used
since `x86_64` built by `runs-on: windows-2022` is working
2024-12-02 11:17:37 -08:00
Bert
1efb9914ee
ci: fix failing python release ( #1896 )
...
Fix failing python release for windows:
https://github.com/lancedb/lancedb/actions/runs/12019637086/job/33506642964
Also updates pkginfo to fix twine build as suggested here:
https://github.com/pypi/warehouse/issues/15611
failing release:
https://github.com/lancedb/lancedb/actions/runs/12091344173/job/33719622146
2024-12-02 11:05:29 -08:00
Mr. Doge
d496ab13a0
ci: linux: specify target triple for neon pack-build (vectordb) ( #1889 )
...
fixes that all `neon pack-build` packs are named
`vectordb-linux-x64-musl-*.tgz` even when cross-compiling
adds 2nd param:
`TARGET_TRIPLE=${2:-x86_64-unknown-linux-gnu}`
`npm run pack-build -- -t $TARGET_TRIPLE`
2024-11-26 10:57:17 -08:00
Will Jones
9fa08bfa93
ci: use correct runner for vectordb ( #1881 )
...
We already do this for `gnu` builds, we should do this also for `musl`
builds.
2024-11-25 16:17:10 -08:00
Mr. Doge
53d1535de1
ci: musl x64,arm64 ( #1853 )
...
untested 4 artifacts at:
https://github.com/FuPeiJiang/lancedb/actions/runs/11926579058
node-native-linux-aarch64-musl 22.6 MB
node-native-linux-x86_64-musl 23.6 MB
nodejs-native-linux-aarch64-musl 26.7 MB
nodejs-native-linux-x86_64-musl 27 MB
this follows the same process as:
https://github.com/lancedb/lancedb/pull/1816#issuecomment-2484816669
Closes #1388
Closes #1107
---------
Co-authored-by: Will Jones <willjones127@gmail.com >
2024-11-20 10:53:19 -08:00
Will Jones
97d6210c33
ci: remove invalid references ( #1834 )
...
Fix release job
2024-11-18 11:32:44 -08:00
Will Jones
587c0824af
feat: flexible null handling and insert subschemas in Python ( #1827 )
...
* Test that we can insert subschemas (omit nullable columns) in Python.
* More work is needed to support this in Node. See:
https://github.com/lancedb/lancedb/issues/1832
* Test that we can insert data with nullable schema but no nulls in
non-nullable schema.
* Add `"null"` option for `on_bad_vectors` where we fill with null if
the vector is bad.
* Make null values not considered bad if the field itself is nullable.
2024-11-15 11:33:00 -08:00
Will Jones
119d88b9db
ci: disable Windows Arm64 until the release builds work ( #1833 )
...
Started to actually fix this, but it was taking too long
https://github.com/lancedb/lancedb/pull/1831
2024-11-14 15:04:23 -08:00
Will Jones
0fd8a50bd7
ci(node): run examples in CI ( #1796 )
...
This is done as setup for a PR that will fix the OpenAI dependency
issue.
* [x] FTS examples
* [x] Setup mock openai
* [x] Ran `npm audit fix`
* [x] sentences embeddings test
* [x] Double check formatting of docs examples
2024-11-13 11:10:56 -08:00
Umut Hope YILDIRIM
9f228feb0e
ci: remove cache to fix build issues on windows arm runner ( #1820 )
2024-11-13 09:27:10 -08:00
Will Jones
68974a4e06
ci: add index URL to fix failing docs build ( #1823 )
2024-11-12 16:54:22 -08:00
Lei Xu
4c9bab0d92
fix: use pandas with pydantic embedding column ( #1818 )
...
* Make Pandas `DataFrame` works with embedding function + Subset of
columns
* Make `lancedb.create_table()` work with embedding function
2024-11-11 14:48:56 -08:00
Umut Hope YILDIRIM
729718cb09
fix: arm64 runner proto already installed bug ( #1810 )
...
https://github.com/lancedb/lancedb/actions/runs/11748512661/job/32732745458
2024-11-08 14:49:37 -08:00
Umut Hope YILDIRIM
b1c84e0bda
feat: added lancedb and vectordb release ci for win32-arm64-msvc npmjs only ( #1805 )
2024-11-08 11:40:57 -08:00
Umut Hope YILDIRIM
fa9ca8f7a6
ci: arm64 windows build support ( #1770 )
...
Adds support for 'aarch64-pc-windows-msvc'.
2024-11-06 15:34:23 -08:00
Lei Xu
f0e7f5f665
ci: change to use github runner ( #1708 )
...
Use github runner
2024-09-27 17:53:05 -07:00
Will Jones
f5c25b6fff
ci: run clippy on tests ( #1659 )
2024-09-23 07:33:47 -07:00
LuQQiu
e118c37228
ci: enable java auto release ( #1602 )
...
Enable bump java pom.xml versions
Enable auto java release when detect stable github release
2024-09-19 10:51:03 -07:00
Lei Xu
4ee7225e91
ci: public java package ( #1485 )
...
Co-authored-by: Lu Qiu <luqiujob@gmail.com >
2024-09-05 11:48:48 -07:00
Rithik Kumar
632007d0e2
docs: add recommender system example ( #1561 )
...
before:
After:
---------
Co-authored-by: Ayush Chaurasia <ayush.chaurarsia@gmail.com >
2024-08-25 12:30:30 +05:30
Gagan Bhullar
277b753fd8
fix: run java stages in parallel ( #1472 )
...
This PR is for issue - https://github.com/lancedb/lancedb/issues/1331
2024-07-27 12:04:32 -07:00
Cory Grinstead
391fa26175
feat(rust): huggingface sentence-transformers ( #1447 )
...
Co-authored-by: Will Jones <willjones127@gmail.com >
2024-07-22 13:47:57 -05:00
Lei Xu
c9c61eb060
docs: expose merge_insert doc for remote python SDK ( #1464 )
...
`merge_insert` API is not shown up on
[`RemoteTable`](https://lancedb.github.io/lancedb/python/saas-python/#lancedb.remote.table.RemoteTable )
today
* Also bump `ruff` version as well
2024-07-22 10:48:16 -07:00
Will Jones
d564f6eacb
ci: fix vectordb release process ( #1450 )
...
* Labelled jobs `vectordb` and `lancedb` so it's clear which package
they are for
* Fix permission issue in aarch64 Linux `vectordb` build that has been
blocking release for two months.
* Added Slack notifications for failure of these publish jobs.
2024-07-17 11:17:33 -07:00
Lei Xu
e780b2f51c
ci: fix nodejs doc test ( #1419 )
...
Fixed nodejs doctest failures due to compiling JNI node.
2024-07-01 10:21:41 -07:00
Weston Pace
ea86dad4b7
feat: upgrade lance to 0.12.2-beta.2 ( #1381 )
2024-06-14 05:43:26 -07:00
Weston Pace
007f9c1af8
chore: change build machine for linux arm ( #1360 )
2024-06-06 13:22:58 -07:00
Weston Pace
1e85b57c82
ci: don't update package locks if we are not releasing node ( #1323 )
...
This doesn't actually block a python-only release since this step runs
after the version bump has been pushed but it still would be nice for
the git job to finish successfully.
2024-05-30 04:42:06 -07:00
LuQQiu
db712b0f99
feat(java): add table names java api ( #1279 )
...
Add lancedb-jni and table names API
---------
Co-authored-by: Lei Xu <eddyxu@gmail.com >
2024-05-24 11:49:11 -07:00
Weston Pace
e4dac751e7
chore: remove working-directory from pypi upload step ( #1322 )
...
The wheels are built to `WORKDIR/target/wheels` and the step was
configured to look for them at `WORKDIR/python/target/wheels`.
2024-05-23 10:31:32 -07:00
Rob Meng
2e197ef387
feat: upgrade lance to 0.11.0 ( #1317 )
...
upgrade lance and make fixes for the upgrade
2024-05-21 18:53:19 -04:00
