docs: show stateful class Functions

The Functions page only showed a plain function, so it never said where a class's `__init__` runs. Users loading a large model need to know it is not the client: registration reads the signature and ships the source.

This adds a section on class Functions: `__init__` runs once per instance on the worker, then `__call__` per row or batch and `close()` on retirement, state is per instance, and `__init__` parameters are initialization fields set per binding. The example's test asserts that registering the class does not construct it.
This commit is contained in:
Wyatt Alt
2026-10-29 04:41:23 +08:00
committed by GitHub
parent 057a39a1c7
commit b4cbaf2f67
3 changed files with 89 additions and 0 deletions
+44
View File
@@ -50,6 +50,25 @@ def describe(value: float):
return {"half": value / 2, "note": None}
# --8<-- [start:function_class_define]
@udf(pip=["sentence-transformers"])
class embed:
def __init__(self, model: str = "all-MiniLM-L6-v2"):
# Runs on the worker, once per instance: only the worker needs the package.
from sentence_transformers import SentenceTransformer
self.model = SentenceTransformer(model)
def __call__(self, text: str) -> list[float]:
return self.model.encode(text).tolist()
def close(self):
del self.model
# --8<-- [end:function_class_define]
# A Function version as the service returns it after registration: the SDK's
# shared golden reply, with the name and signature of the definition above.
SERVICE_REPLY = (
@@ -166,6 +185,31 @@ def test_function_version_binds_a_column(tmp_path):
table.add_columns({"doubled": application})
def test_class_function_is_constructed_only_where_it_runs():
# Registration reads __init__'s signature; it never constructs the class,
# so the model's package need not be installed here.
assert "sentence_transformers" not in sys.modules
signature = embed.registration_request.signature
assert [
(field.name, field.arrow_type, field.nullable)
for field in signature.initialization
] == [("model", "utf8", True)]
assert [(p.name, p.arrow_type) for p in signature.inputs] == [("text", "utf8")]
assert "sentence_transformers" not in sys.modules
embedder = registered_version(embed)
# --8<-- [start:function_class_apply]
from lancedb import col
application = embedder(text=col("text"), model="all-mpnet-base-v2")
# --8<-- [end:function_class_apply]
assert application.initialization == {"model": "all-mpnet-base-v2"}
# The default applies to an omitted field.
assert embedder(text=col("text")).initialization == {}
with pytest.raises(TypeError, match="unknown arguments"):
embedder(text=col("text"), device="cuda")
def test_function_enterprise(monkeypatch):
require_deployment()
+41
View File
@@ -19,6 +19,8 @@ import {
PyFunctionApply as FunctionApply,
PyFunctionRefresh as FunctionRefresh,
PyFunctionInspect as FunctionInspect,
PyFunctionClassDefine as FunctionClassDefine,
PyFunctionClassApply as FunctionClassApply,
} from '/snippets/function.mdx';
import { PyConnectEnterprisePlaceholders as FunctionConnect } from '/snippets/connection.mdx';
@@ -143,6 +145,45 @@ rows added since the last one and recomputes rows whose inputs changed, and
leaves filled rows with unchanged inputs alone. See
[Refresh a column](/feature-engineering/refresh-column).
## Keep state in a class {#classes}
A function that loads a model or opens a client should not do it for every
row. Decorate a class instead: its `__init__` does the expensive setup once,
and `__call__` handles each row or batch.
<CodeBlock filename="Python" language="Python" icon="python">
{FunctionClassDefine}
</CodeBlock>
Registering the class does not construct it. `@udf` reads the signature of
`__init__` and packages the source, so the client never loads the model. On
the cluster, each instance of the function calls `__init__` once. It then calls
`__call__` for every row or batch it processes, and `close()`, if defined, once
when it retires. Import heavy packages inside `__init__`, as above, so only the
worker needs them installed, and declare them in `pip`.
State belongs to one instance. An instance reuses it across every batch it
serves, but instances do not share it, so each one holds its own copy of the
model. Plan memory, or a GPU, for one copy per instance.
The annotated parameters of `__init__` are the function's initialization
fields. Their values come from each binding rather than from the function, so
one registered version can run with different settings on different columns:
<CodeBlock filename="Python" language="Python" icon="python">
{FunctionClassApply}
</CodeBlock>
Pass the application to `add_columns` as [above](#apply). Initialization values
are fixed for every row of a binding and take booleans, integers, floats,
strings, and lists or structs of them. A parameter with a default may be
omitted. Initialization values are stored with the binding, so keep
credentials out of them: pass `secrets=[EnvVarSecret(...)]` to
`create_function` and read the variable in `__init__`.
Calling the decorated class in your own process constructs it, the same as an
undecorated class, so local tests can create an instance and call it directly.
## Read and produce blobs {#blobs}
A Function can bind a [Blob v2 column](/management/table#managed-blobs) and
+4
View File
@@ -2,6 +2,10 @@
export const PyFunctionApply = "from lancedb import col\n\ntable.add_columns({\"doubled\": version(value=col(\"value\"))})\n";
export const PyFunctionClassApply = "from lancedb import col\n\napplication = embedder(text=col(\"text\"), model=\"all-mpnet-base-v2\")\n";
export const PyFunctionClassDefine = "@udf(pip=[\"sentence-transformers\"])\nclass embed:\n def __init__(self, model: str = \"all-MiniLM-L6-v2\"):\n # Runs on the worker, once per instance: only the worker needs the package.\n from sentence_transformers import SentenceTransformer\n\n self.model = SentenceTransformer(model)\n\n def __call__(self, text: str) -> list[float]:\n return self.model.encode(text).tolist()\n\n def close(self):\n del self.model\n";
export const PyFunctionConnect = "import os\n\nimport lancedb\n\ndb = lancedb.connect(\n uri=os.environ[\"LANCEDB_URI\"], # db://your-database\n api_key=os.environ[\"LANCEDB_API_KEY\"],\n region=os.environ.get(\"LANCEDB_REGION\", \"us-east-1\"),\n host_override=os.environ[\"LANCEDB_HOST_OVERRIDE\"],\n)\n";
export const PyFunctionCreateTable = "import pyarrow as pa\n\nschema = pa.schema([pa.field(\"value\", pa.float64(), nullable=False)])\ntable = db.create_table(\n \"function_demo\",\n [{\"value\": 1.0}, {\"value\": 2.0}, {\"value\": 3.0}],\n schema=schema,\n)\n";