feat(python): bundle Function package resources (#4987)

A package shipped through `code=` currently loses its non-Python files.
This breaks vendored packages that read JSON configuration, manifests,
prompts, or binary resources after deployment. Include regular package
resources in a separate base64-encoded `python_bundle.resources` map,
preserving exact bytes and relative paths. Both source and resources
contribute to the artifact digest. UTF-8 Python source bytes retain
their original line endings as well, so manifests that hash vendor code
continue to match.

For example, `@function(code=[my_package])` can use
`importlib.resources.files("my_package").joinpath("models.json").read_text()`
on the worker. Hidden files and caches are excluded; symlinks and unsafe
paths warn instead of following files outside the package. Source-only
bundles retain their existing wire form. Resource bundles require
matching server support.

Adds an isolated-import regression with a vendored subpackage, JSON
metadata, binary data, SHA256 verification, preserved CRLF bytes, and
resource-only version changes. Local builds and tests are disabled by
request. Ruff formatting and linting run locally; CI runs the tests.
This commit is contained in:
Jack Ye
2026-11-13 16:21:58 +08:00
committed by GitHub
parent db098ba812
commit bab22a3fbd
4 changed files with 187 additions and 58 deletions
+12
View File
@@ -1420,3 +1420,15 @@ name; omitting it allows matching reads of any revision. Prefixes must be concre
paths without wildcards or traversal. This integration does not make Hugging Face
a writable database location for `ATTACH DATABASE`. Dataset SQL sources select a
matching integration automatically, with no integration argument in the query.
### Function package resources
`@function(code=[my_package])` includes the package’s Python modules and data
files, including vendored subpackages. Resource paths and bytes are preserved,
so `importlib.resources` and package integrity checks work after deployment.
Changing a resource changes the Function artifact digest. This requires a server
that accepts `python_bundle.resources`.
Caches, hidden paths, symlinks and unsafe paths are excluded. Use `pip=` or
`conda=` for platform-specific dependencies and `models=` for large model snapshots.
+14 -7
View File
@@ -499,13 +499,20 @@ def caption(jpeg: bytes) -> str:
</CodeGroup>
Each module in `code` is captured and shipped the same way the function
itself is: as source, snapshotted at registration. Only `.py` source files are
included; package data such as JSON, YAML, prompt, or tokenizer files is not
copied. `@function` warns before registration when it skips such files from a
package in `code`, naming the affected paths. To use a package that includes
the required resources, install it with `pip=` or `conda=` and remove that
package from `code`; otherwise the Python-only copied package shadows the
installed distribution.
itself is: as source, snapshotted at registration. Packages also include regular
data files such as JSON, YAML, prompts, and tokenizer files, including resources
inside vendored subpackages. Resource paths and bytes are preserved for
`importlib.resources` and package integrity checks. Changing a resource changes
the Function artifact's identity. Caches, hidden paths, symlinks, and unsafe
paths are excluded.
Package resources require matching SDK and server support. Upgrade query nodes
and Function image-building workers before registering resource bundles; older
servers reject them. Older SDKs ship only Python source and warn when excluding
package data. On deployments without resource-bundle support, install the
package with `pip=` or `conda=` and omit it from `code` so the copied package does
not shadow the installed distribution. Use environment dependencies for
platform-specific packages and `models=` for large model snapshots.
## Drop it {#drop}