mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-16 07:36:50 +00:00
feat(python): bundle Function package resources (#4987)
A package shipped through `code=` currently loses its non-Python files.
This breaks vendored packages that read JSON configuration, manifests,
prompts, or binary resources after deployment. Include regular package
resources in a separate base64-encoded `python_bundle.resources` map,
preserving exact bytes and relative paths. Both source and resources
contribute to the artifact digest. UTF-8 Python source bytes retain
their original line endings as well, so manifests that hash vendor code
continue to match.
For example, `@function(code=[my_package])` can use
`importlib.resources.files("my_package").joinpath("models.json").read_text()`
on the worker. Hidden files and caches are excluded; symlinks and unsafe
paths warn instead of following files outside the package. Source-only
bundles retain their existing wire form. Resource bundles require
matching server support.
Adds an isolated-import regression with a vendored subpackage, JSON
metadata, binary data, SHA256 verification, preserved CRLF bytes, and
resource-only version changes. Local builds and tests are disabled by
request. Ruff formatting and linting run locally; CI runs the tests.
This commit is contained in:
@@ -1420,3 +1420,15 @@ name; omitting it allows matching reads of any revision. Prefixes must be concre
|
||||
paths without wildcards or traversal. This integration does not make Hugging Face
|
||||
a writable database location for `ATTACH DATABASE`. Dataset SQL sources select a
|
||||
matching integration automatically, with no integration argument in the query.
|
||||
|
||||
|
||||
### Function package resources
|
||||
|
||||
`@function(code=[my_package])` includes the package’s Python modules and data
|
||||
files, including vendored subpackages. Resource paths and bytes are preserved,
|
||||
so `importlib.resources` and package integrity checks work after deployment.
|
||||
Changing a resource changes the Function artifact digest. This requires a server
|
||||
that accepts `python_bundle.resources`.
|
||||
|
||||
Caches, hidden paths, symlinks and unsafe paths are excluded. Use `pip=` or
|
||||
`conda=` for platform-specific dependencies and `models=` for large model snapshots.
|
||||
|
||||
Reference in New Issue
Block a user