mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-14 15:52:26 +00:00
feat(secrets): named Secrets, bindings, and namespace addressing (#4150)
Adds the client half of database-scoped named Secrets: a Secret is a
name and
an opaque value stored by the service, and a Function binds one to the
environment variable its library already reads. Secrets are addressed by
a
namespace path plus a name.
The UDF body is unchanged and stays portable — it reads `OPENAI_API_KEY`
the
way it always did, and the binding is what puts a value there:
```python
db.create_secret("openai-prod", os.environ["OPENAI_API_KEY"])
function = db.create_function(
analyze_caption,
secrets=[
EnvVarSecret(secret_name="openai-prod", env_variable="OPENAI_API_KEY")
],
)
function.secret_bindings # the Secret's name, never its value
```
- `create_secret` / `alter_secret` / `list_secrets` / `describe_secret`
/
`drop_secret` on sync, async and remote connections, with the pyo3
binding
and the Rust client behind them. Each takes `namespace_path`
keyword-only,
defaulting to the root.
- **There is no read API, by construction rather than by policy** — no
code
path returns a stored credential, and `describe_secret` answers with
metadata only.
- `EnvVarSecret` is a pure local constructor: it contacts no server, so
it
cannot fail on a Secret that does not exist. It exists so that a bare
string
in that position — which would be a credential — is a `TypeError` rather
than a plausible-looking mistake that reads identically in a diff.
- `create_function(..., secrets=[...])` carries the bindings as
`secret_bindings`: a list of `SecretBinding` tagged by `kind`, so a
later
delivery mode is a variant rather than a sibling field. The value never
travels — it is resolved by the service when the Function runs, which is
what
lets a rotation reach columns already pinned to an older
FunctionVersion.
- A binding names its Secret as a `SecretReference` of `{name,
namespace_path}`
rather than one joined string, so no delimiter has to be excluded from
every
name and segment forever, and `ClientConfig.id_delimiter` cannot
contradict
an identity built on a fixed separator.
- A root namespace is omitted from the request body rather than sent
empty, so
a root request is byte-identical to one from a client that predates
namespaces. Tests pin it.
This is the client surface the design's §4 describes; the service side
lives in
sophon.
**Previously split across two PRs.** Namespace addressing was #4151,
stacked on
this one; it is folded in here so the Secret identity contract — name,
namespace path, and the binding that carries both — is reviewable as one
piece
rather than as a shape introduced and then replaced.
## Identifier safety, merged from #4189
**#4189 is merged into this branch**, so the client half of Secrets and
the
guards on the identity it puts in the URL are one PR. What it added:
- Components are checked where the identifier is built, before a request
is
constructed. `create_secret("../jobs", value)` no longer resolves to
`/v1/jobs/create` and delivers a credential-bearing body to a route with
none
of this one's body suppression.
- Each component is percent-encoded and joined by the delimiter, so
nothing
inside a component can end the path segment or add one.
- A component may not be empty, a relative segment (`.`, `..`, and their
`%2e`
spellings), or the delimiter itself — the three ways a component erases
a
boundary the split has to recover. `["prod", ""]` joined to `prod$`,
which
reads back as `["prod"]`.
- `$` is the only accepted `id_delimiter`, refused at client
construction.
`ClientConfig.id_delimiter` remains, since the identifier grammar comes
from
the Lance REST catalog standard, but a value that would produce
identifiers no
service splits the caller's way is now an error where it was written.
- One `build_object_identifier` and one character set serve tables,
namespaces,
Secrets, Functions and materialized views.
Components are checked for *addressability*, not a character set: the
name's
own grammar stays each object's own, so a catalog database keeps the `/`
that
`RemoteCatalog::validate_name` allows.
## Known shortcoming
`secret_bindings` is omitted from a registration body when empty, so a
client
that binds nothing sends what a client without bindings sends. When a
client
does bind a Secret and the service does not know the field, the field is
ignored: registration succeeds, the returned version carries no
bindings, and
the Function fails at execution with the variable unset, far from the
call that
asked for it.
`ServerVersion` is how this codebase refuses a feature the service is
too old
for, and it gates five features already. It does not gate this one: it
is held
per table, and registering a Function is a database-level call. Noted at
the
field in `remote/db.rs`; wiring the gate is follow-up work.
**Tests:** lancedb lib 1340 passed, `first_class_function_slice1` 9,
`first_class_function_slice2` 3, plus Python tests across both slices.
Rebased onto `main` after #4176 (OCI Function identity), #4191 (`.`/`..`
table
names) and #4195 (remote catalogs).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01UfmeJ533rQDnPBkMtjerV6
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
f8d73b3447
commit
60a1b4c219
@@ -91,7 +91,7 @@ def test_catalog_sync_scope_and_serialization(catalog_server):
|
||||
assert requests[0][0] == "/v1/namespace/team%2Fsearch/create"
|
||||
assert requests[0][2] == {"mode": "ExistOk"}
|
||||
assert requests[1][0] == "/v1/namespace/team%2Fsearch/describe"
|
||||
assert requests[4][0] == "/v1/namespace/%24/list?limit=1&page_token=a%2Fb"
|
||||
assert requests[4][0] == "/v1/namespace/$/list?limit=1&page_token=a%2Fb"
|
||||
assert requests[5][2] == {"mode": "Skip", "behavior": "Restrict"}
|
||||
for i, (_, headers, _) in enumerate(requests):
|
||||
headers = {key.lower(): value for key, value in headers.items()}
|
||||
|
||||
@@ -13,7 +13,9 @@ from lancedb.functions import (
|
||||
FunctionBinding,
|
||||
FunctionVersion,
|
||||
PythonRuntimeSpec,
|
||||
SecretBinding,
|
||||
RefreshColumnResult,
|
||||
SecretReference,
|
||||
)
|
||||
from lancedb.table import AsyncTable
|
||||
|
||||
@@ -37,6 +39,22 @@ def job_result(name: str) -> dict:
|
||||
return json.loads(fixture(name))["result"]
|
||||
|
||||
|
||||
def assert_no_secret_values(value):
|
||||
"""No client value models a resolved credential, at any nesting depth."""
|
||||
if isinstance(value, dict):
|
||||
for key, child in value.items():
|
||||
assert key not in {
|
||||
"secret_value",
|
||||
"secret_values",
|
||||
"resolved_secret",
|
||||
"resolved_secrets",
|
||||
}
|
||||
assert_no_secret_values(child)
|
||||
elif isinstance(value, list):
|
||||
for child in value:
|
||||
assert_no_secret_values(child)
|
||||
|
||||
|
||||
def test_public_function_values_are_in_api_reference():
|
||||
docs = Path(__file__).parents[3] / "docs" / "src" / "python" / "python.md"
|
||||
rendered = docs.read_text()
|
||||
@@ -96,6 +114,11 @@ def test_function_version_identity_is_immutable_and_exact():
|
||||
assert version.version == "1"
|
||||
assert version.image.manifest_digest.startswith("sha256:")
|
||||
assert version.version != version.image.manifest_digest
|
||||
assert list(version.secret_bindings) == [
|
||||
SecretBinding(
|
||||
kind="env", variable="HF_TOKEN", secret_ref=SecretReference(name="hf-prod")
|
||||
)
|
||||
]
|
||||
|
||||
with pytest.raises((TypeError, ValueError)):
|
||||
version.version = "1"
|
||||
@@ -282,6 +305,31 @@ def test_refresh_result_rejects_non_u64_values(field):
|
||||
RefreshColumnResult.from_json(json.dumps(value))
|
||||
|
||||
|
||||
def test_canonical_client_values_carry_bindings_and_no_credentials():
|
||||
"""A binding names a Secret; the credential behind it has no client field."""
|
||||
version = FunctionVersion.from_json(
|
||||
json.dumps(job_result("remote_function_job.json"))
|
||||
)
|
||||
canonical = json.loads(version.to_canonical_json())
|
||||
assert canonical["secret_bindings"] == [
|
||||
{"kind": "env", "variable": "HF_TOKEN", "secret_ref": {"name": "hf-prod"}}
|
||||
]
|
||||
assert_no_secret_values(canonical)
|
||||
|
||||
|
||||
def test_a_version_without_bindings_omits_the_field_in_both_directions():
|
||||
"""A Function that binds nothing carries no ``secret_bindings`` key.
|
||||
|
||||
Absent decodes as an empty list, and an empty list serializes back to
|
||||
absent.
|
||||
"""
|
||||
value = job_result("remote_function_job.json")
|
||||
del value["secret_bindings"]
|
||||
version = FunctionVersion.from_json(json.dumps(value))
|
||||
assert list(version.secret_bindings) == []
|
||||
assert "secret_bindings" not in json.loads(version.to_canonical_json())
|
||||
|
||||
|
||||
class _FunctionDeclarationInner:
|
||||
def __init__(self):
|
||||
self.calls = []
|
||||
|
||||
@@ -11,6 +11,7 @@ import types
|
||||
from datetime import date
|
||||
import http.server
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
@@ -24,11 +25,14 @@ import pytest
|
||||
import lancedb
|
||||
from lancedb.functions import (
|
||||
PythonRuntimeSpec,
|
||||
SecretBinding,
|
||||
SecretReference,
|
||||
UdfDefinition,
|
||||
_canonical_arrow_type,
|
||||
_GRAMMAR_PRIMITIVES,
|
||||
udf,
|
||||
)
|
||||
from lancedb.secrets import EnvVarSecret
|
||||
|
||||
THRESHOLD = 20
|
||||
_CACHE = None
|
||||
@@ -59,6 +63,15 @@ def normalize_score(value: float) -> float:
|
||||
return value / 100.0
|
||||
|
||||
|
||||
@udf(
|
||||
pip=["openai==3.7.0"],
|
||||
env={"MODE": "test"},
|
||||
python_version="3.12",
|
||||
)
|
||||
def analyze_caption(caption: str) -> str:
|
||||
return caption.strip()
|
||||
|
||||
|
||||
def test_scalar_udf_matches_shared_registration_golden_and_remains_callable():
|
||||
assert isinstance(normalize_score, UdfDefinition)
|
||||
assert normalize_score(25.0) == 0.25
|
||||
@@ -75,6 +88,286 @@ def test_scalar_udf_matches_shared_registration_golden_and_remains_callable():
|
||||
}
|
||||
|
||||
|
||||
def test_secret_bound_udf_matches_its_shared_registration_golden():
|
||||
assert analyze_caption(" hello ") == "hello"
|
||||
bound = analyze_caption.bind_secrets(
|
||||
[EnvVarSecret(secret_name="openai-prod", env_variable="OPENAI_API_KEY")]
|
||||
)
|
||||
assert (
|
||||
bound.to_canonical_json()
|
||||
== (FIXTURES / "remote_function_secret_registration_request.canonical.json")
|
||||
.read_text()
|
||||
.strip()
|
||||
)
|
||||
|
||||
|
||||
def test_a_namespaced_binding_records_the_path_and_the_name():
|
||||
"""A binding names the parts, so nothing has to be parsed back out.
|
||||
|
||||
A root binding carries no path at all: the field is absent rather than an
|
||||
empty list, so a binding states a namespace only when it has one.
|
||||
"""
|
||||
root = EnvVarSecret(secret_name="openai-prod", env_variable="OPENAI_API_KEY")
|
||||
assert root.secret_namespace_path == []
|
||||
|
||||
nested = EnvVarSecret(
|
||||
secret_name="openai-prod",
|
||||
env_variable="OPENAI_API_KEY",
|
||||
secret_namespace_path=["prod", "vision"],
|
||||
)
|
||||
assert nested.secret_namespace_path == ["prod", "vision"]
|
||||
assert nested != root
|
||||
|
||||
bound = analyze_caption.bind_secrets([nested])
|
||||
assert list(bound.secret_bindings) == [
|
||||
SecretBinding(
|
||||
kind="env",
|
||||
variable="OPENAI_API_KEY",
|
||||
secret_ref=SecretReference(
|
||||
name="openai-prod", namespace_path=("prod", "vision")
|
||||
),
|
||||
)
|
||||
]
|
||||
|
||||
at_root = analyze_caption.bind_secrets([root])
|
||||
assert list(at_root.secret_bindings) == [
|
||||
SecretBinding(
|
||||
kind="env",
|
||||
variable="OPENAI_API_KEY",
|
||||
secret_ref=SecretReference(name="openai-prod"),
|
||||
)
|
||||
]
|
||||
# A root binding carries no path at all on the wire.
|
||||
canonical = json.loads(at_root.to_canonical_json())
|
||||
assert canonical["secret_bindings"] == [
|
||||
{
|
||||
"kind": "env",
|
||||
"variable": "OPENAI_API_KEY",
|
||||
"secret_ref": {"name": "openai-prod"},
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_a_namespace_path_is_validated_locally():
|
||||
# The charset is the service's, not a delimiter's: a reference is never
|
||||
# joined, so a segment cannot make anything parse two ways.
|
||||
with pytest.raises(ValueError):
|
||||
EnvVarSecret(
|
||||
secret_name="openai-prod",
|
||||
env_variable="K",
|
||||
secret_namespace_path=["with$delim"],
|
||||
)
|
||||
with pytest.raises(ValueError):
|
||||
EnvVarSecret(
|
||||
secret_name="openai-prod", env_variable="K", secret_namespace_path=["a/b"]
|
||||
)
|
||||
# A bare string is a plausible mistake with the wrong meaning.
|
||||
with pytest.raises(TypeError):
|
||||
EnvVarSecret(
|
||||
secret_name="openai-prod", env_variable="K", secret_namespace_path="prod"
|
||||
)
|
||||
|
||||
|
||||
def test_an_unbound_request_carries_no_binding_at_all():
|
||||
"""Binding is a registration-time decision, so the definition holds none.
|
||||
|
||||
The decorator declares nothing about secrets, which is what makes the PRD's
|
||||
claim true: a Function's source and its registration request are identical
|
||||
whether or not a credential is later bound to it.
|
||||
"""
|
||||
unbound = json.loads(analyze_caption.registration_request.to_canonical_json())
|
||||
assert "secret_bindings" not in unbound
|
||||
assert "OPENAI_API_KEY" not in json.dumps(unbound)
|
||||
|
||||
|
||||
def test_binding_a_secret_leaves_the_packaged_artifact_untouched():
|
||||
"""The artifact is source bytes and nothing else, with or without secrets."""
|
||||
bound = analyze_caption.bind_secrets(
|
||||
[EnvVarSecret(secret_name="openai-prod", env_variable="OPENAI_API_KEY")]
|
||||
)
|
||||
assert bound.artifact == analyze_caption.registration_request.artifact
|
||||
assert bound.artifact.digest == analyze_caption.registration_request.artifact.digest
|
||||
|
||||
|
||||
def test_a_function_declaring_no_secret_is_registered_exactly_as_before():
|
||||
"""The compatibility claim: nothing about the no-secret path moves."""
|
||||
assert (
|
||||
normalize_score.bind_secrets(None).to_canonical_json()
|
||||
== normalize_score.registration_request.to_canonical_json()
|
||||
)
|
||||
assert (
|
||||
"secret_bindings"
|
||||
not in normalize_score.registration_request.to_canonical_json()
|
||||
)
|
||||
|
||||
|
||||
def test_a_function_binds_each_variable_once():
|
||||
with pytest.raises(ValueError, match="binds each environment variable once"):
|
||||
analyze_caption.bind_secrets(
|
||||
[
|
||||
EnvVarSecret(secret_name="openai-prod", env_variable="OPENAI_API_KEY"),
|
||||
EnvVarSecret(
|
||||
secret_name="openai-staging", env_variable="OPENAI_API_KEY"
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def test_bindings_may_not_collide_with_plain_configuration():
|
||||
"""`env` is stored with the Function; a Secret is not. Refuse, do not pick."""
|
||||
with pytest.raises(ValueError, match="must be disjoint"):
|
||||
analyze_caption.bind_secrets(
|
||||
[EnvVarSecret(secret_name="mode-prod", env_variable="MODE")]
|
||||
)
|
||||
|
||||
|
||||
def test_a_binding_envelope_reaches_the_service_for_it_to_judge():
|
||||
"""Binding rules are the service's: it owns the runtime the names land in.
|
||||
|
||||
The client sends what it was given, so a rule it duplicated could disagree
|
||||
with the service's without either side noticing. What is checked here is
|
||||
that the envelope arrives intact -- the shape the service judges is the
|
||||
shape the caller wrote.
|
||||
"""
|
||||
with _mock_remote_function_catalog() as (host, state):
|
||||
db = lancedb.connect(
|
||||
"db://dev",
|
||||
api_key="fake",
|
||||
host_override=host,
|
||||
client_config={"retry_config": {"retries": 0}},
|
||||
)
|
||||
bindings = [
|
||||
EnvVarSecret(secret_name=f"secret-{index}", env_variable=f"TOKEN_{index}")
|
||||
for index in range(17)
|
||||
]
|
||||
db.create_function(normalize_score, secrets=bindings)
|
||||
|
||||
sent = state["requests"][0][1]
|
||||
assert len(sent["secret_bindings"]) == 17
|
||||
assert {
|
||||
"kind": "env",
|
||||
"variable": "TOKEN_0",
|
||||
"secret_ref": {"name": "secret-0"},
|
||||
} in sent["secret_bindings"]
|
||||
|
||||
|
||||
_SECRET_DEBUG_LOG_SOURCE = """
|
||||
import http.server
|
||||
import json
|
||||
import threading
|
||||
|
||||
import lancedb
|
||||
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
def do_POST(self):
|
||||
self.rfile.read(int(self.headers.get("Content-Length", "0")))
|
||||
payload = json.dumps({}).encode()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(payload)))
|
||||
self.end_headers()
|
||||
self.wfile.write(payload)
|
||||
|
||||
|
||||
server = http.server.ThreadingHTTPServer(("localhost", 0), Handler)
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
try:
|
||||
db = lancedb.connect(
|
||||
"db://dev",
|
||||
api_key="API_KEY_SENTINEL",
|
||||
host_override="http://localhost:%d" % server.server_address[1],
|
||||
client_config={"retry_config": {"retries": 0}},
|
||||
)
|
||||
db.create_secret("openai-prod", "SECRET_VALUE_SENTINEL")
|
||||
finally:
|
||||
server.shutdown()
|
||||
"""
|
||||
|
||||
|
||||
def test_a_credential_never_reaches_a_debug_log(tmp_path):
|
||||
"""The logger sees the serialized body, so no value-side redaction reaches it.
|
||||
|
||||
Runs in a subprocess because the Rust logger reads ``LANCEDB_LOG`` once, at
|
||||
import.
|
||||
"""
|
||||
script = tmp_path / "write_secret.py"
|
||||
script.write_text(_SECRET_DEBUG_LOG_SOURCE)
|
||||
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(script)],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env={**os.environ, "LANCEDB_LOG": "debug"},
|
||||
)
|
||||
output = result.stdout + result.stderr
|
||||
|
||||
# Without this the test passes when debug logging is simply off.
|
||||
assert "Sending request_id=" in output, output
|
||||
assert "SECRET_VALUE_SENTINEL" not in output
|
||||
assert "API_KEY_SENTINEL" not in output
|
||||
|
||||
|
||||
def test_a_credential_value_is_rejected_in_the_binding_position():
|
||||
"""The one mistake the typed binding exists to stop."""
|
||||
with pytest.raises(TypeError, match="EnvVarSecret"):
|
||||
analyze_caption.bind_secrets(["sk-live-0001"])
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("secret", "variable", "message"),
|
||||
[
|
||||
("openai-prod", "not-a-var", "invalid environment variable name"),
|
||||
("openai-prod", "API-TOKEN", "invalid environment variable name"),
|
||||
("not a name", "API_TOKEN", "invalid Secret name"),
|
||||
("openai$prod", "API_TOKEN", "invalid Secret name"),
|
||||
],
|
||||
)
|
||||
def test_a_binding_validates_both_names_locally(secret, variable, message):
|
||||
with pytest.raises(ValueError, match=message):
|
||||
EnvVarSecret(secret_name=secret, env_variable=variable)
|
||||
|
||||
|
||||
def test_a_secret_name_admits_what_a_namespace_name_does():
|
||||
"""A Secret has to be nameable wherever a namespace already is.
|
||||
|
||||
LanceDB namespace and table names are `[A-Za-z0-9_.-]` with no rule about
|
||||
which character comes first, so a name may lead with `_`, `-` or `.`.
|
||||
Anything narrower here would leave Secrets unaddressable inside namespaces
|
||||
that already exist -- the reason periods are admitted is the reason the
|
||||
edges are too.
|
||||
"""
|
||||
for name in ["openai.prod.v1", ".hidden", "_internal", "-lead", "trailing."]:
|
||||
binding = EnvVarSecret(secret_name=name, env_variable="OPENAI_API_KEY")
|
||||
assert binding.secret_name == name
|
||||
|
||||
for name in ["", "with/slash", "with$delimiter", "a" * 256]:
|
||||
with pytest.raises(ValueError, match="invalid Secret name"):
|
||||
EnvVarSecret(secret_name=name, env_variable="OPENAI_API_KEY")
|
||||
|
||||
# A namespace segment follows the same rule, and LanceDB already admits
|
||||
# these shapes as namespace names -- so a Secret is addressable inside one.
|
||||
for segment in [".hidden", "_internal", "-lead", "trailing."]:
|
||||
binding = EnvVarSecret(
|
||||
secret_name="openai-prod",
|
||||
env_variable="OPENAI_API_KEY",
|
||||
secret_namespace_path=[segment],
|
||||
)
|
||||
assert binding.secret_namespace_path == [segment]
|
||||
|
||||
for segment in ["", "with/slash", "with$delimiter"]:
|
||||
with pytest.raises(ValueError, match="invalid namespace path segment"):
|
||||
EnvVarSecret(
|
||||
secret_name="openai-prod",
|
||||
env_variable="OPENAI_API_KEY",
|
||||
secret_namespace_path=[segment],
|
||||
)
|
||||
|
||||
|
||||
def _main_udf_source(
|
||||
*, threshold: int = 20, input_annotation: str = "int", comparison: str = ">="
|
||||
) -> str:
|
||||
@@ -1232,7 +1525,20 @@ def _mock_remote_function_catalog():
|
||||
body = json.loads(self.rfile.read(length) or b"{}")
|
||||
state["requests"].append((self.path, body))
|
||||
status = 200
|
||||
if self.path == "/v1/function/normalize_score/create":
|
||||
# `{id}` is the Function name, so match on the shape rather than on
|
||||
# one name: these tests register more than one Function.
|
||||
parts = self.path.strip("/").split("/")
|
||||
function_action = (
|
||||
(urllib.parse.unquote(parts[2]), parts[3])
|
||||
if len(parts) == 4 and parts[0] == "v1" and parts[1] == "function"
|
||||
else (None, None)
|
||||
)
|
||||
secret_action = (
|
||||
(urllib.parse.unquote(parts[2]), parts[3])
|
||||
if len(parts) == 4 and parts[0] == "v1" and parts[1] == "secret"
|
||||
else (None, None)
|
||||
)
|
||||
if function_action[1] == "create":
|
||||
state["version"] = {
|
||||
"name": "normalize_score",
|
||||
"version": FUNCTION_VERSION,
|
||||
@@ -1246,6 +1552,7 @@ def _mock_remote_function_catalog():
|
||||
).read_text()
|
||||
)["image"],
|
||||
"signature": body["signature"],
|
||||
"secret_bindings": body.get("secret_bindings", []),
|
||||
"created_at": "2026-08-21T00:00:00Z",
|
||||
}
|
||||
response = {"job_id": "job-register"}
|
||||
@@ -1264,6 +1571,15 @@ def _mock_remote_function_catalog():
|
||||
elif self.path == "/v1/function/normalize_score/drop":
|
||||
assert body == {"version": FUNCTION_VERSION}
|
||||
response = {"dropped": True}
|
||||
elif secret_action[1] in ("create", "alter"):
|
||||
# The Secret is the path identifier, so the body is the value.
|
||||
assert set(body) == {"value"}
|
||||
assert secret_action[0] == "openai-prod"
|
||||
response = {}
|
||||
elif secret_action[1] == "drop":
|
||||
assert secret_action[0] == "openai-prod"
|
||||
assert body == {}
|
||||
response = {}
|
||||
else:
|
||||
status = 404
|
||||
response = {"error": "not found"}
|
||||
@@ -1276,6 +1592,16 @@ def _mock_remote_function_catalog():
|
||||
for key, values in urllib.parse.parse_qs(url.query).items()
|
||||
}
|
||||
state["requests"].append((url.path, query))
|
||||
if url.path == "/v1/namespace/$/secret/list":
|
||||
if "page_token" not in query:
|
||||
self._write_response(
|
||||
200,
|
||||
{"secrets": [{"name": "openai-prod"}], "page_token": "next"},
|
||||
)
|
||||
else:
|
||||
assert query["page_token"] == "next"
|
||||
self._write_response(200, {"secrets": [{"name": "hf-prod"}]})
|
||||
return
|
||||
if url.path != "/v1/namespace/$/function/list":
|
||||
self._write_response(404, {"error": "not found"})
|
||||
return
|
||||
@@ -1330,6 +1656,92 @@ def test_remote_registration_job_and_exact_version_reopen_round_trip():
|
||||
assert create_request == expected_request
|
||||
|
||||
|
||||
def test_remote_registration_sends_bindings_and_never_a_credential():
|
||||
with _mock_remote_function_catalog() as (host, state):
|
||||
db = lancedb.connect(
|
||||
"db://dev",
|
||||
api_key="fake",
|
||||
host_override=host,
|
||||
client_config={"retry_config": {"retries": 0}},
|
||||
)
|
||||
created = db.create_function(
|
||||
analyze_caption,
|
||||
secrets=[
|
||||
EnvVarSecret(secret_name="openai-prod", env_variable="OPENAI_API_KEY")
|
||||
],
|
||||
)
|
||||
|
||||
assert list(created.secret_bindings) == [
|
||||
SecretBinding(
|
||||
kind="env",
|
||||
variable="OPENAI_API_KEY",
|
||||
secret_ref=SecretReference(name="openai-prod"),
|
||||
)
|
||||
]
|
||||
path, create_request = state["requests"][0]
|
||||
assert path == "/v1/function/analyze_caption/create"
|
||||
assert create_request["secret_bindings"] == [
|
||||
{
|
||||
"kind": "env",
|
||||
"variable": "OPENAI_API_KEY",
|
||||
"secret_ref": {"name": "openai-prod"},
|
||||
}
|
||||
]
|
||||
# The request names a Secret and carries nothing that could be one. The
|
||||
# Function's own name is the path identifier rather than a body field, so
|
||||
# it is the one key the body does not repeat.
|
||||
expected = json.loads(
|
||||
analyze_caption.bind_secrets(
|
||||
[EnvVarSecret(secret_name="openai-prod", env_variable="OPENAI_API_KEY")]
|
||||
).to_canonical_json()
|
||||
)
|
||||
assert expected.pop("name") == "analyze_caption"
|
||||
assert create_request == expected
|
||||
|
||||
|
||||
def test_remote_secret_verbs_round_trip():
|
||||
with _mock_remote_function_catalog() as (host, state):
|
||||
db = lancedb.connect(
|
||||
"db://dev",
|
||||
api_key="fake",
|
||||
host_override=host,
|
||||
client_config={"retry_config": {"retries": 0}},
|
||||
)
|
||||
assert db.create_secret("openai-prod", "sk-live-0001") is None
|
||||
assert db.alter_secret("openai-prod", "sk-live-0002") is None
|
||||
assert db.list_secrets() == ["openai-prod", "hf-prod"]
|
||||
assert db.drop_secret("openai-prod") is None
|
||||
|
||||
routes = [path for path, _ in state["requests"]]
|
||||
assert routes == [
|
||||
"/v1/secret/openai-prod/create",
|
||||
"/v1/secret/openai-prod/alter",
|
||||
"/v1/namespace/$/secret/list",
|
||||
"/v1/namespace/$/secret/list",
|
||||
"/v1/secret/openai-prod/drop",
|
||||
]
|
||||
# The Secret is the path identifier, so the body is the value alone.
|
||||
assert state["requests"][0][1] == {"value": "sk-live-0001"}
|
||||
# Listing is a GET: the first page asks for nothing, the second resumes on
|
||||
# the token the server handed back, and neither carries a body.
|
||||
assert state["requests"][2][1] == {}
|
||||
assert state["requests"][3][1] == {"page_token": "next"}
|
||||
|
||||
|
||||
def test_building_a_binding_contacts_no_server():
|
||||
"""A binding is a local value: it says nothing about whether the Secret exists.
|
||||
|
||||
Existence is the server's answer at registration, where a mistyped name is a
|
||||
clear error rather than a client-side check that was already stale.
|
||||
"""
|
||||
with _mock_remote_function_catalog() as (_host, state):
|
||||
binding = EnvVarSecret(secret_name="openai-prod", env_variable="OPENAI_API_KEY")
|
||||
assert binding.secret_name == "openai-prod"
|
||||
assert binding.env_variable == "OPENAI_API_KEY"
|
||||
|
||||
assert state["requests"] == []
|
||||
|
||||
|
||||
def test_blocking_remote_registration_returns_function_version():
|
||||
with _mock_remote_function_catalog() as (host, state):
|
||||
db = lancedb.connect(
|
||||
|
||||
Reference in New Issue
Block a user