mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 14:16:54 +00:00
fix(python): explain Function namespace import restrictions (#4982)
A plain `import ctypes.util` in a Function currently reports “dynamic
namespace access” without explaining the module restriction or a way
forward. Give blocked module imports a distinct error that names the
modules, explains the reconstructed namespace constraint, and recommends
an importable helper supplied through `code=`, `pip=`, or `conda=`.
Dynamic namespace operations also get an actionable explanation.
Document the full restrictions and the helper alternative in the
decorator and Python Functions reference. Add regression coverage for
direct imports, `from` imports, global module aliases, class Functions,
the deprecated `@udf` alias, and helpers imported normally or shipped
with `code=`. Existing import restrictions remain in place.
For example, move `ctypes.util.find_library` into
`native_diagnostics/__init__.py`, then use:
```python
import native_diagnostics
from lancedb import function
@function(code=[native_diagnostics])
def probe_library(name: str) -> str:
return native_diagnostics.find_library(name)
```
Fixes ENT-3408.
This commit is contained in:
@@ -447,6 +447,51 @@ assert scaler(3.0) == 6.0
|
||||
scaler.close()
|
||||
```
|
||||
|
||||
### Function packaging restrictions
|
||||
|
||||
The decorator snapshots the Function's source and referenced globals into a new
|
||||
module. It conservatively rejects imports of `sys`, `builtins`, `importlib`,
|
||||
`inspect`, `gc`, `ctypes`, and `types`, including submodules such as `ctypes.util`
|
||||
and `importlib.metadata`. This applies to imports inside function or class bodies
|
||||
and references to module-level aliases. Even ordinary uses such as `sys.version`
|
||||
and `ctypes.util.find_library` are blocked: these modules can reach names outside
|
||||
the reconstructed Function namespace. References to `globals`, `locals`, `vars`,
|
||||
`eval`, `exec`, `compile`, and `__import__` are rejected as dynamic namespace access.
|
||||
These restrictions also apply to the deprecated `@udf` alias.
|
||||
|
||||
For legitimate uses, move the code into an importable helper module and ship it
|
||||
with `code=`, or install its package with `pip=` or `conda=`. Imported helpers keep
|
||||
their own namespace; the packager does not scan their bodies. Helpers defined in
|
||||
`__main__` are copied by source and have the same restrictions as the Function.
|
||||
|
||||
For example, put this helper in `native_diagnostics/__init__.py`:
|
||||
|
||||
```python
|
||||
import ctypes.util
|
||||
|
||||
|
||||
def find_library(name: str) -> str:
|
||||
return ctypes.util.find_library(name) or ""
|
||||
```
|
||||
|
||||
Then import the helper and include its source in the Function's artifact:
|
||||
|
||||
```python
|
||||
import native_diagnostics
|
||||
from lancedb import function
|
||||
|
||||
|
||||
@function(code=[native_diagnostics])
|
||||
def probe_library(name: str) -> str:
|
||||
return native_diagnostics.find_library(name)
|
||||
```
|
||||
|
||||
Captured literal globals support `None`, booleans, integers, strings, bytes,
|
||||
finite floats, and tuples of these values. Construct other objects, including
|
||||
PyArrow types and schemas, inside the Function or helper module instead. Values
|
||||
used only in the decorator's `input_schema` or `output_schema` are not captured
|
||||
globals and can be defined outside the Function.
|
||||
|
||||
Annotate a blob input with `lancedb.BlobReader` for lazy, seekable access, or
|
||||
with `bytes` to materialize its contents before the Function runs. Reader
|
||||
inputs infer the Blob V2 type without an explicit input schema. Readers are
|
||||
|
||||
Reference in New Issue
Block a user