A Function's remote environment can now be conda instead of pip.
`@udf(conda=[...], conda_channels=[...])` registers one; pip and conda
are exclusive, channels are priority-ordered and require conda. The Rust
and Python `PythonEnvironmentSpec` models gain `channels`, dropped from
the canonical JSON when empty so existing pip registrations keep their
digests.
## Problem
The unreleased First-Class Function authoring API exposed
`secrets=[...]` and serialized `required_secrets`, promising runtime
resolution and injection that Sophon does not implement.
## Behavior
Remove the secrets dimension from the public Python decorator, Python
and Rust registration/version models, and shared wire fixtures. Existing
successfully registered Functions retain stable identity: the field
could only be empty and empty values were already omitted from canonical
serialization.
A stable LanceDB release has not shipped this Function API, so this
contracts the surface before it becomes a published compatibility
commitment.
## Validation gap
The Rust shared-golden Function suites and Python formatting/lint checks
pass locally. Python pytest was not run because the local environment
lacks its runtime dependencies and the frozen native editable build did
not complete in practical time.
Registering a real (embedding) Function failed on the client for three
reasons:
- `_package_source` treated `inspect.getclosurevars().unbound` as
"unresolved globals"; CPython puts attribute names there, so any body
with `np.linalg.norm(...)` or `body.split()` was rejected. Module-scope
references now come from Python's own scope analysis (`symtable`) over
the function source, recursively, and each is resolved the way the
interpreter would: the function's globals first (a module global may
shadow a builtin), then builtins. Free variables of nested scopes stay
lexical; postponed annotations are not runtime loads. A genuinely
missing global still fails.
- `_canonical_arrow_type` emitted spellings the server's frozen grammar
rejects (`fixed_size_list<T>[n]`, `timestamp[us]`, `struct<...>`,
zero-sized lists). It now emits exactly the grammar, with the server's
`fixed_size_list<item, size>` form, and the Rust declaration planner
parses that form too.
A shared golden
(`tests/fixtures/first_class_functions/v1/arrow_types.json`)
enumerates every grammar type, nested forms and rejected spellings; the
Python emitter and Rust parser are tested against it, and the same file
is under test in sophon. Packaging tests execute the shipped artifact in
a fresh namespace.
Contract changes (hence `breaking-change`):
- `@udf` now rejects namespace acquisition structurally
(`globals()`/`eval`/... by name, plus `import
sys`/`builtins`/`importlib`/`inspect` inside the body), requires the
function's captured `__builtins__` to be the standard mapping itself
(identity, so neither lookups nor implicit hooks such as `__import__`
can differ), rejects module globals that are namespace-bearing modules
(`builtins`, `sys`, ...), and treats the function's own name as
recursion only when the module binds it to the function or to the exact
`UdfDefinition` the decorator produced; it resolves module globals
through the function's real namespace (a module global may shadow a
builtin) and ships importable classes/functions as imports.
- List outputs must declare a non-nullable, metadata-free child named
`item` (`pa.list_(pa.field("item", t, nullable=False))`); that is what
the grammar means, and pyarrow's default nullable child was being
silently collapsed into it.
Contract, stated in the `udf` docstring: the artifact is a snapshot of
the function source plus exactly the module names it references.
Reaching the module namespace by another route is rejected where a
static packager can see it and is otherwise unsupported; there is no
dynamic-access detection beyond that.
## Problem
`refresh_column_async` returned a unit-result job even though durable
refresh jobs carry a canonical terminal result. Python callers could not
obtain row counts or source and published versions through the public
`Job` API, and local and remote refresh jobs exposed different result
semantics.
## Behavior
`refresh_column_async` now returns `Job[RefreshColumnResult]` for local
and remote tables. The general typed-job bridge binds each endpoint to
its public result model while preserving unit-result jobs and existing
status, wait, cancel, and timeout behavior. A local no-op refresh
reports no published version.
The Node.js API continues to resolve `wait()` as `void`; its binding
erases the Rust result type internally to preserve the existing public
contract.
## Ownership and integration boundary
LanceDB owns the language-neutral `Job<T>` contract and language-binding
decode. Sophon owns production and durable persistence of terminal
payloads. Sophon #7348 and #7378 now publish the canonical refresh
result for Function-backed and expression-backed refresh jobs,
respectively. The remote client fixture matches the merged server
schema; live deployment and end-to-end demo acceptance remain separate
rollout checks.
Remote Function catalog requests used singular endpoints that are not
exposed by Phalanx. Route registration to `POST /v1/functions/create`
and exact-version lookup to `POST /v1/functions/get`, while preserving
the existing typed Job submission and wait behavior.
## Problem
The canonical Function wire values and typed remote Job contract do not
yet provide a Python authoring surface or catalog client, so users
cannot package a scalar callable, register it, or reopen the exact
immutable Function version.
## Behavior
This adds scalar-only `@udf` authoring with deterministic annotation or
explicit Arrow schema validation, content-addressed Python artifacts,
and an internal scalar-to-Arrow-batch adapter descriptor. Registration
payloads model non-secret environment values and secret names only.
Remote connections can submit `create_function_async` and receive a
typed `Job<FunctionVersion>`, then reopen that exact version by name and
version ID. Synchronous connections can call `create_function` to submit
and wait for the immutable version in one operation. Local Function
catalog operations return a stable `NotSupported` error. Shared
Rust/Python golden payloads and mocked catalog responses freeze the
request, typed terminal result, and exact lookup contract.
## Validation
- Rust formatting, remote check, clippy, and focused LDB-1/LDB-2 tests
- Python formatting, lint, and focused LDB-1/LDB-2 tests
- Python API documentation build
Wyatt Alt
Xuanwo