Compare commits

...
Author SHA1 Message Date
Jonathan M HsiehandClaude Opus 5 e9586850b6 fix(secrets): admit periods in Secret names and namespace segments
LanceDB namespace and table names are `[A-Za-z0-9_.-]`, so excluding periods
here put Secrets out of reach inside any namespace a user already has one in --
unreachable to create, alter, describe, drop or bind, with no way to recover
but renaming the namespace.

The rule is that set and nothing further. No positional constraint rides along:
the established grammar says nothing about which character comes first, so a
segment may begin with `_`, `-` or `.` today, and a Secret has to be nameable
wherever a namespace already is. A narrower rule would reintroduce the same
unaddressability it is here to remove, one character class over.

Names and path segments follow one rule, and the client matches the service.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UfmeJ533rQDnPBkMtjerV6
2026-09-11 15:32:05 +00:00
Jonathan M HsiehandClaude Opus 5 e541a3be93 fix(secrets): complete the binding and value contract at the Rust boundary
`validate` checked only the binding count, so a caller reaching the low-level
entry point past `bind_secrets` could register an environment variable name the
runtime cannot deliver, or bind a Secret to a name `runtime.env` already sets --
which would resolve by delivery order, with a value visible in the Function's
record and a value that is not.

Secret values are bounded here too, at the limit the service enforces, so an
oversized credential is refused before a request body is built rather than
after it has been serialized and uploaded.

Sandbox-reserved names are deliberately still the service's alone: that list
belongs to the runtime that owns it, and a copy here would drift from it
silently.

Both gate reproducers now fail closed with no request reaching the service.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UfmeJ533rQDnPBkMtjerV6
2026-09-09 17:33:59 +00:00
Jonathan M HsiehandClaude Opus 5 b85f5f141f fix(secrets): keep credentials out of the client's own debug log
`log_request` logs any JSON body verbatim at debug, and Python and Node both
wire that logger to `LANCEDB_LOG`. `create_secret` posts the value in its body,
so ordinary SDK debug logging wrote the credential to application logs. The
comment on `write_secret` reasoned correctly about proxy traces and access logs
and missed the logger in this process.

Redaction cannot live in the value model: the logger sees the serialized body,
where the credential is already plaintext bytes. So bodies are suppressed for
the route instead, matched on the `secrets` path segment rather than a
versioned prefix, so a verb added under that namespace later is covered without
an edit here.

The same debug line prints the request's Debug, which prints headers, so the
API key was in every debug line of every request regardless of route. Marking
the header value sensitive is what stops that.

The end-to-end regression fails without this: the log carried
`,"value":"SECRET_VALUE_SENTINEL"}` verbatim.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UfmeJ533rQDnPBkMtjerV6
2026-09-09 16:59:40 +00:00
Jonathan M HsiehandClaude Opus 5 4d09f8ce26 refactor(secrets): name the binding field for its delivery mode, cap in Rust
`secret_bindings` read as the general record of what a Function needs, but a
`Map<String, String>` keyed on environment-variable name can only ever hold an
env-delivered binding: an accessor binding has no variable to key on. Naming
the field for its delivery mode leaves a later mode a sibling field rather than
a tagged value type, which would break a field already in the identity hash.

The per-Function cap moves from `bind_secrets` to
`Connection::create_function_async`, above the backend dispatch, so it holds
for every language surface rather than only the one that validates first. The
low-level PyO3 entry point reached the wire past the Python check; it no longer
does.

Also corrects a doc comment claiming registration fails when a bound Secret is
absent. Existence is first answered at `add_columns`, by design.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UfmeJ533rQDnPBkMtjerV6
2026-09-09 16:59:40 +00:00
Jonathan M HsiehandClaude Opus 5 7b29fb2f51 feat(secrets): named Secrets and EnvVarSecret bindings
Adds the client half of database-scoped named Secrets: a Secret is a name and
an opaque value stored by the service, and a Function binds one to the
environment variable its library already reads.

- `db.create_secret` / `alter_secret` / `list_secrets` / `describe_secret` /
  `drop_secret` on sync, async and remote connections, with the pyo3 binding
  and the Rust client behind them. There is no read API by construction rather
  than by policy: no code path returns a stored credential, and
  `describe_secret` answers with metadata only.
- `EnvVarSecret(secret=..., env_variable=...)` pairs a Secret with the variable
  it arrives in. It is a pure local constructor -- it contacts no server, so it
  cannot fail on a Secret that does not exist -- and it exists so a bare string
  in that position, which would be a credential, is a TypeError rather than a
  plausible-looking mistake that reads identically in a diff.
- `create_function(..., secrets=[...])` carries the bindings as
  `secret_bindings`, a map from variable name to Secret name. The value never
  travels: it is resolved by the service when the Function runs, which is what
  lets a rotation reach columns pinned to an older FunctionVersion.

The UDF body is unchanged and stays portable -- it reads `OPENAI_API_KEY` the
way it always did, and the binding is what puts a value there.

Squashed: the original three commits were a first design plus a rewrite of it,
so their sequence describes an interface that no longer exists.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XE1UwYKsgbb3USBfkqCE6v
2026-09-09 00:12:57 +00:00
Lance Release 0111a72dc3 Bump version: 0.39.0-beta.3 → 0.39.0-beta.4 2026-09-06 20:46:55 +00:00
LanceDB RobotandJack Ye a487d4033e chore: update lance dependency to v12.0.0-beta.14 (#4141)
Update the Rust workspace Lance dependencies and Java lance-core from
v12.0.0-beta.11 to
[v12.0.0-beta.14](https://github.com/lance-format/lance/releases/tag/v12.0.0-beta.14),
refreshing Cargo.lock. Resolve two Clippy diagnostics by making an
internal Node.js helper private and using a byte string literal in a
remote-table test fixture.

Validation: `cargo clippy --quiet --workspace --tests --all-features --
-D warnings`, `cargo fmt --all --quiet`, `git diff --check`, and `pnpm
build` in nodejs.

---------

Co-authored-by: Jack Ye <yezhaoqin@gmail.com>
2026-09-06 13:43:35 -07:00
dependabot[bot] 02ea0dda9f build(deps-dev): bump the nodejs-deps group across 1 directory with 2 updates (#4134)
Bumps the nodejs-deps group with 2 updates in the /nodejs directory:
[@opentelemetry/sdk-metrics](https://github.com/open-telemetry/opentelemetry-js)
and [ts-jest](https://github.com/kulshekhar/ts-jest).

Updates `@opentelemetry/sdk-metrics` from 2.10.0 to 2.11.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/open-telemetry/opentelemetry-js/releases">@​opentelemetry/sdk-metrics's
releases</a>.</em></p>
<blockquote>
<h2>v2.11.0</h2>
<h2>2.11.0</h2>
<h3>🚀 Features</h3>
<ul>
<li>feat(context-async-hooks): implement <code>attach()</code> on
<code>AsyncLocalStorageContextManager</code> <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6845">#6845</a>
<a href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a>
<ul>
<li>On Node.js 25.9+, delegates to
<code>AsyncLocalStorage.withScope()</code> returning a native
<code>RunScope</code>. On older Node.js, falls back to
<code>enterWith()</code> with a manual disposable wrapper.</li>
</ul>
</li>
<li>feat(sdk-trace): allow configuring the force flush timeout per call
<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6929">#6929</a>
<a
href="https://github.com/LarryHu0217"><code>@​LarryHu0217</code></a></li>
</ul>
<h3>🐛 Bug Fixes</h3>
<ul>
<li>fix(sdk-metrics): ignore <code>Infinity</code> in exponential
histograms <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7015">#7015</a>
<a href="https://github.com/mwear"><code>@​mwear</code></a></li>
</ul>
<h3>🏠 Internal</h3>
<ul>
<li>perf(sdk-metrics): reuse a single DataView for exponential histogram
bit reads <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6998">#6998</a>
<a href="https://github.com/mwear"><code>@​mwear</code></a></li>
<li>chore(ci): run documentation tests on a weekly schedule <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6920">#6920</a>
<a
href="https://github.com/LarryHu0217"><code>@​LarryHu0217</code></a></li>
<li>feat(ci): support pre-releases and major version bumps in the
release workflow <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6768">#6768</a>
<a
href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a></li>
<li>chore(resources): Ensure that multiple uses of
serviceInstanceIdDetector.detect() return the <em>same</em> value for
<code>service.instance.id</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/open-telemetry/opentelemetry-js/blob/main/CHANGELOG.md">@​opentelemetry/sdk-metrics's
changelog</a>.</em></p>
<blockquote>
<h2>2.11.0</h2>
<h3>🚀 Features</h3>
<ul>
<li>feat(context-async-hooks): implement <code>attach()</code> on
<code>AsyncLocalStorageContextManager</code> <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6845">#6845</a>
<a href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a>
<ul>
<li>On Node.js 25.9+, delegates to
<code>AsyncLocalStorage.withScope()</code> returning a native
<code>RunScope</code>. On older Node.js, falls back to
<code>enterWith()</code> with a manual disposable wrapper.</li>
</ul>
</li>
<li>feat(sdk-trace): allow configuring the force flush timeout per call
<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6929">#6929</a>
<a
href="https://github.com/LarryHu0217"><code>@​LarryHu0217</code></a></li>
</ul>
<h3>🐛 Bug Fixes</h3>
<ul>
<li>fix(sdk-trace-base): avoid a Webpack self-reference error in
CommonJS output <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6981">#6981</a>
<a href="https://github.com/sansynx"><code>@​sansynx</code></a></li>
<li>fix(sdk-metrics): ignore <code>Infinity</code> in exponential
histograms <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/7015">#7015</a>
<a href="https://github.com/mwear"><code>@​mwear</code></a></li>
</ul>
<h3>🏠 Internal</h3>
<ul>
<li>perf(sdk-metrics): reuse a single DataView for exponential histogram
bit reads <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6998">#6998</a>
<a href="https://github.com/mwear"><code>@​mwear</code></a></li>
<li>chore(ci): run documentation tests on a weekly schedule <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/pull/6920">#6920</a>
<a
href="https://github.com/LarryHu0217"><code>@​LarryHu0217</code></a></li>
<li>feat(ci): support pre-releases and major version bumps in the
release workflow <a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6768">#6768</a>
<a
href="https://github.com/pichlermarc"><code>@​pichlermarc</code></a></li>
<li>chore(resources): Ensure that multiple uses of
serviceInstanceIdDetector.detect() return the <em>same</em> value for
<code>service.instance.id</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/0b72a81636fa476e8f1f1afd2ae0c90a1362194c"><code>0b72a81</code></a>
chore: prepare next release (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7044">#7044</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/a9c5338a9f485f1433df9308f24bd7397a4a0321"><code>a9c5338</code></a>
ci: roll prerelease changelog into one final release changelog (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7045">#7045</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/f41805e769ba10fb6dae72a4b7a5a3dc67cca82e"><code>f41805e</code></a>
chore: prepare next release (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7042">#7042</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/b85eb28343ff6234e2bb4d46b7b4a3d360e5ea2f"><code>b85eb28</code></a>
chore(instrumentation-http): fix lint errors (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7039">#7039</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/3f9253009be2ae48419432e79a597ead7be8be6a"><code>3f92530</code></a>
ci: support pre-releases and major version bumps in release workflow (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7035">#7035</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/82a58316a63b6fde62afd268e145b82222d328cf"><code>82a5831</code></a>
docs(otlp-exporter-base): document HTTP exporter options (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/6735">#6735</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/e086dec7f9304107ef6d50b5877be88895c06aa7"><code>e086dec</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/59dac70d00d46fa56b2b921cf721fd922730f23d"><code>59dac70</code></a>
chore(deps): update jamesives/github-pages-deploy-action action to
v4.9.0 (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7">#7</a>...</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/d0ce7532b058631ec9eec111c04fefe7fd873e1f"><code>d0ce753</code></a>
chore: add <a
href="https://github.com/maryliag"><code>@​maryliag</code></a> to
maintainers (<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7024">#7024</a>)</li>
<li><a
href="https://github.com/open-telemetry/opentelemetry-js/commit/03469a129f97265c8eec93d566e9e00d4f741db3"><code>03469a1</code></a>
chore(deps): update open-telemetry/shared-workflows action to v0.10.0
(<a
href="https://redirect.github.com/open-telemetry/opentelemetry-js/issues/7032">#7032</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/open-telemetry/opentelemetry-js/compare/v2.10.0...v2.11.0">compare
view</a></li>
</ul>
</details>
<br />

Updates `ts-jest` from 29.4.9 to 29.4.12
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/kulshekhar/ts-jest/releases">ts-jest's
releases</a>.</em></p>
<blockquote>
<h2>v29.4.12</h2>
<p>Please refer to <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v29.4.11</h2>
<p>Please refer to <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
<h2>v29.4.10</h2>
<p>Please refer to <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">CHANGELOG.md</a>
for details.</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/kulshekhar/ts-jest/blob/main/CHANGELOG.md">ts-jest's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.11...v29.4.12">29.4.12</a>
(2026-07-22)</h2>
<h3>Features</h3>
<ul>
<li><strong>compiler:</strong> support TypeScript 7 projects through
compatibility aliases (<a
href="https://redirect.github.com/kulshekhar/ts-jest/pull/5386">#5386</a>)</li>
</ul>
<h2><a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.10...v29.4.11">29.4.11</a>
(2026-05-21)</h2>
<h3>Bug Fixes</h3>
<ul>
<li>preserve Bundler on the CJS path under TypeScript &gt;= 6 (<a
href="https://github.com/kulshekhar/ts-jest/commit/39418187515f11b6584d35a4e3ddf50231f74936">3941818</a>),
closes <a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/4198">#4198</a></li>
</ul>
<h2><a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.9...v29.4.10">29.4.10</a>
(2026-05-18)</h2>
<h3>Bug Fixes</h3>
<ul>
<li>pass <code>resolutionMode</code> to
<code>ts.resolveModuleName</code> for hybrid module support (<a
href="https://github.com/kulshekhar/ts-jest/commit/b557a85f85c3fd34523ec3a15293afbdc9dea83c">b557a85</a>)</li>
<li>rebuild <code>Program</code> when consecutive compiles need
different module kinds (<a
href="https://github.com/kulshekhar/ts-jest/commit/a82a2b32c4987a5249fd5284283117dd2fa3be47">a82a2b3</a>),
closes <a
href="https://redirect.github.com/kulshekhar/ts-jest/issues/4774">#4774</a></li>
<li>respect tsconfig <code>moduleResolution</code> instead of forcing
<code>Node10</code> (<a
href="https://github.com/kulshekhar/ts-jest/commit/1bffffc667557c173ae0c1f93dd436920775dac4">1bffffc</a>)</li>
<li><strong>transformer:</strong> transpile <code>mjs</code> files from
<code>node_modules</code> for CJS mode (<a
href="https://github.com/kulshekhar/ts-jest/commit/96d025dd912ea2bceb18b67d2d509ada7a756d9d">96d025d</a>)</li>
<li><strong>transformer:</strong> use a consistent comparator in
hoist-jest sortStatements (<a
href="https://github.com/kulshekhar/ts-jest/commit/8a8fd2fb8446655bba18367db9306a1089490e62">8a8fd2f</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/3f05625da10da954fdf0a10394385008275ddbb3"><code>3f05625</code></a>
chore(release): 29.4.12</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/df28b27f2e60edf275866763a3cdf745360d3eae"><code>df28b27</code></a>
docs: clarify TypeScript version prerequisites</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/c8a614af419b1dcc90f3d1a7a48238ac1b637e6b"><code>c8a614a</code></a>
docs: mention TypeScript 7 setup in README</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/06c79d4cebfa785749b2f96ef2dbeffc12798c47"><code>06c79d4</code></a>
fix: address TypeScript 7 review feedback</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/f10746008e512203ad7d8c581e2c58cc7dcd43c8"><code>f107460</code></a>
docs: explain TypeScript 7 compatibility setup</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/33882274c1e4fc3a06cece3b21f4873182c8fee7"><code>3388227</code></a>
test(e2e): add TypeScript compatibility matrix</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/891dc731ff9f31785ad9698e4d7cfa6078991fe6"><code>891dc73</code></a>
fix(compiler): support TypeScript 7 compatibility aliases</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/eb135ebe819991b1e10c998915cc6db2057c4de1"><code>eb135eb</code></a>
build(deps-dev): bump shell-quote from 1.8.4 to 1.10.0 in /examples</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/d5d80a34bc575be130db2b9f5cd0958173982cda"><code>d5d80a3</code></a>
ci: pin google osv scan action at v2.3.5</li>
<li><a
href="https://github.com/kulshekhar/ts-jest/commit/6bf293f0a4ddf468735d81fcdf04f923278e030c"><code>6bf293f</code></a>
build(deps): bump shell-quote from 1.8.4 to 1.10.0 in /website</li>
<li>Additional commits viewable in <a
href="https://github.com/kulshekhar/ts-jest/compare/v29.4.9...v29.4.12">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-06 09:41:24 -07:00
43 changed files with 1886 additions and 231 deletions
+1 -1
View File
@@ -1,5 +1,5 @@
[tool.bumpversion]
current_version = "0.39.0-beta.3"
current_version = "0.39.0-beta.4"
parse = """(?x)
(?P<major>0|[1-9]\\d*)\\.
(?P<minor>0|[1-9]\\d*)\\.
Generated
+49 -47
View File
@@ -3526,8 +3526,8 @@ checksum = "42703706b716c37f96a77aea830392ad231f44c9e9a67872fa5548707e11b11c"
[[package]]
name = "fsst"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-array",
"rand 0.9.5",
@@ -4886,8 +4886,8 @@ checksum = "e037a2e1d8d5fdbd49b16a4ea09d5d6401c1f29eca5ff29d03d3824dba16256a"
[[package]]
name = "lance"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arc-swap",
"arrow",
@@ -4959,8 +4959,8 @@ dependencies = [
[[package]]
name = "lance-arrow"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -4982,7 +4982,7 @@ dependencies = [
[[package]]
name = "lance-arrow-scalar"
version = "58.0.0"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -4996,7 +4996,7 @@ dependencies = [
[[package]]
name = "lance-arrow-stats"
version = "58.0.0"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-array",
"arrow-schema",
@@ -5005,8 +5005,8 @@ dependencies = [
[[package]]
name = "lance-bitpacking"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrayref",
"crunchy",
@@ -5016,8 +5016,8 @@ dependencies = [
[[package]]
name = "lance-core"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -5054,8 +5054,8 @@ dependencies = [
[[package]]
name = "lance-datafusion"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow",
"arrow-array",
@@ -5085,8 +5085,8 @@ dependencies = [
[[package]]
name = "lance-datagen"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow",
"arrow-array",
@@ -5103,8 +5103,8 @@ dependencies = [
[[package]]
name = "lance-derive"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"proc-macro2",
"quote",
@@ -5113,8 +5113,8 @@ dependencies = [
[[package]]
name = "lance-encoding"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-arith",
"arrow-array",
@@ -5147,8 +5147,8 @@ dependencies = [
[[package]]
name = "lance-file"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-arith",
"arrow-array",
@@ -5179,8 +5179,8 @@ dependencies = [
[[package]]
name = "lance-index"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arc-swap",
"arrow",
@@ -5244,8 +5244,8 @@ dependencies = [
[[package]]
name = "lance-index-core"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-array",
"arrow-schema",
@@ -5267,8 +5267,8 @@ dependencies = [
[[package]]
name = "lance-io"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow",
"arrow-array",
@@ -5308,8 +5308,8 @@ dependencies = [
[[package]]
name = "lance-linalg"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-array",
"arrow-schema",
@@ -5323,21 +5323,23 @@ dependencies = [
[[package]]
name = "lance-namespace"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow",
"async-trait",
"bytes",
"lance-core",
"lance-namespace-reqwest-client",
"serde",
"serde_json",
"snafu 0.9.0",
]
[[package]]
name = "lance-namespace-impls"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow",
"arrow-ipc",
@@ -5376,9 +5378,9 @@ dependencies = [
[[package]]
name = "lance-namespace-reqwest-client"
version = "0.11.1"
version = "0.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d06b1fbb5d41f93bc652b61e2872af92e8a6c5f6b4ce8839a8ecfa05365d359"
checksum = "d8d23e54b1634d5bbb434f8dd33dc3c05f6e58d876a9a27b3b4aef58ddbe11af"
dependencies = [
"reqwest 0.12.28",
"serde",
@@ -5390,8 +5392,8 @@ dependencies = [
[[package]]
name = "lance-select"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -5405,8 +5407,8 @@ dependencies = [
[[package]]
name = "lance-table"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow",
"arrow-array",
@@ -5446,8 +5448,8 @@ dependencies = [
[[package]]
name = "lance-testing"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"arrow-array",
"arrow-schema",
@@ -5460,8 +5462,8 @@ dependencies = [
[[package]]
name = "lance-tokenizer"
version = "12.0.0-beta.11"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.11#4a0e26895729feb86d0cb9c09d551bfd619c6472"
version = "12.0.0-beta.14"
source = "git+https://github.com/lance-format/lance.git?tag=v12.0.0-beta.14#a8a101774a1c9647065cc60137094feadbe55296"
dependencies = [
"frostem",
"icu_segmenter",
@@ -5474,7 +5476,7 @@ dependencies = [
[[package]]
name = "lancedb"
version = "0.39.0-beta.3"
version = "0.39.0-beta.4"
dependencies = [
"ahash",
"anyhow",
@@ -5565,7 +5567,7 @@ dependencies = [
[[package]]
name = "lancedb-nodejs"
version = "0.39.0-beta.3"
version = "0.39.0-beta.4"
dependencies = [
"arrow-array",
"arrow-buffer",
@@ -5590,7 +5592,7 @@ dependencies = [
[[package]]
name = "lancedb-python"
version = "0.39.0-beta.3"
version = "0.39.0-beta.4"
dependencies = [
"arrow",
"async-trait",
+14 -14
View File
@@ -13,20 +13,20 @@ categories = ["database-implementations"]
rust-version = "1.91.0"
[workspace.dependencies]
lance = { "version" = "=12.0.0-beta.11", default-features = false, "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-core = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-datagen = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-file = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-io = { "version" = "=12.0.0-beta.11", default-features = false, "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-index = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-linalg = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-namespace = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-namespace-impls = { "version" = "=12.0.0-beta.11", default-features = false, "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-table = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-testing = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-datafusion = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-encoding = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance-arrow = { "version" = "=12.0.0-beta.11", "tag" = "v12.0.0-beta.11", "git" = "https://github.com/lance-format/lance.git" }
lance = { "version" = "=12.0.0-beta.14", default-features = false, "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-core = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-datagen = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-file = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-io = { "version" = "=12.0.0-beta.14", default-features = false, "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-index = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-linalg = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-namespace = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-namespace-impls = { "version" = "=12.0.0-beta.14", default-features = false, "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-table = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-testing = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-datafusion = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-encoding = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lance-arrow = { "version" = "=12.0.0-beta.14", "tag" = "v12.0.0-beta.14", "git" = "https://github.com/lance-format/lance.git" }
lancedb = { path = "rust/lancedb", default-features = false }
ahash = "0.8"
# Note that this one does not include pyarrow
+1 -1
View File
@@ -14,7 +14,7 @@ Add the following dependency to your `pom.xml`:
<dependency>
<groupId>com.lancedb</groupId>
<artifactId>lancedb-core</artifactId>
<version>0.39.0-beta.3</version>
<version>0.39.0-beta.4</version>
</dependency>
```
+4
View File
@@ -125,6 +125,10 @@ listing a storage directory.
::: lancedb.functions.UdfDefinition
::: lancedb.secrets.EnvVarSecret
::: lancedb.secrets.SecretInfo
::: lancedb.functions.FunctionRegistrationRequest
::: lancedb.functions.FunctionArtifactRequest
+1 -1
View File
@@ -8,7 +8,7 @@
<parent>
<groupId>com.lancedb</groupId>
<artifactId>lancedb-parent</artifactId>
<version>0.39.0-beta.3</version>
<version>0.39.0-beta.4</version>
<relativePath>../pom.xml</relativePath>
</parent>
+2 -2
View File
@@ -6,7 +6,7 @@
<groupId>com.lancedb</groupId>
<artifactId>lancedb-parent</artifactId>
<version>0.39.0-beta.3</version>
<version>0.39.0-beta.4</version>
<packaging>pom</packaging>
<name>${project.artifactId}</name>
<description>LanceDB Java SDK Parent POM</description>
@@ -28,7 +28,7 @@
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<arrow.version>15.0.0</arrow.version>
<lance-core.version>12.0.0-beta.11</lance-core.version>
<lance-core.version>12.0.0-beta.14</lance-core.version>
<spotless.skip>false</spotless.skip>
<spotless.version>2.30.0</spotless.version>
<spotless.java.googlejavaformat.version>1.7</spotless.java.googlejavaformat.version>
+1 -1
View File
@@ -1,7 +1,7 @@
[package]
name = "lancedb-nodejs"
edition.workspace = true
version = "0.39.0-beta.3"
version = "0.39.0-beta.4"
publish = false
license.workspace = true
description.workspace = true
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@lancedb/lancedb-darwin-arm64",
"version": "0.39.0-beta.3",
"version": "0.39.0-beta.4",
"os": ["darwin"],
"cpu": ["arm64"],
"main": "lancedb.darwin-arm64.node",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@lancedb/lancedb-linux-arm64-gnu",
"version": "0.39.0-beta.3",
"version": "0.39.0-beta.4",
"os": ["linux"],
"cpu": ["arm64"],
"main": "lancedb.linux-arm64-gnu.node",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@lancedb/lancedb-linux-arm64-musl",
"version": "0.39.0-beta.3",
"version": "0.39.0-beta.4",
"os": ["linux"],
"cpu": ["arm64"],
"main": "lancedb.linux-arm64-musl.node",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@lancedb/lancedb-linux-x64-gnu",
"version": "0.39.0-beta.3",
"version": "0.39.0-beta.4",
"os": ["linux"],
"cpu": ["x64"],
"main": "lancedb.linux-x64-gnu.node",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@lancedb/lancedb-linux-x64-musl",
"version": "0.39.0-beta.3",
"version": "0.39.0-beta.4",
"os": ["linux"],
"cpu": ["x64"],
"main": "lancedb.linux-x64-musl.node",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@lancedb/lancedb-win32-arm64-msvc",
"version": "0.39.0-beta.3",
"version": "0.39.0-beta.4",
"os": [
"win32"
],
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@lancedb/lancedb-win32-x64-msvc",
"version": "0.39.0-beta.3",
"version": "0.39.0-beta.4",
"os": ["win32"],
"cpu": ["x64"],
"main": "lancedb.win32-x64-msvc.node",
+1 -1
View File
@@ -11,7 +11,7 @@
"ann"
],
"private": false,
"version": "0.39.0-beta.3",
"version": "0.39.0-beta.4",
"main": "dist/index.js",
"exports": {
".": "./dist/index.js",
+24 -24
View File
@@ -41,7 +41,7 @@ importers:
version: 3.7.0(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3)(@types/node@22.7.4)
'@opentelemetry/sdk-metrics':
specifier: ^2.10.0
version: 2.10.0(@opentelemetry/api@1.9.1)
version: 2.11.0(@opentelemetry/api@1.9.1)
'@types/axios':
specifier: ^0.14.0
version: 0.14.4
@@ -80,7 +80,7 @@ importers:
version: 0.2.7
ts-jest:
specifier: ^29.1.2
version: 29.4.9(@babel/core@7.29.7)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.29.7))(jest-util@29.7.0)(jest@29.7.0(@types/node@22.7.4))(typescript@5.5.4)
version: 29.4.12(@babel/core@7.29.7)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.29.7))(jest-util@29.7.0)(jest@29.7.0(@types/node@22.7.4))(typescript@5.5.4)
typedoc:
specifier: 0.26.4
version: 0.26.4(typescript@5.5.4)
@@ -1394,20 +1394,20 @@ packages:
resolution: {integrity: sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==}
engines: {node: '>=8.0.0'}
'@opentelemetry/core@2.10.0':
resolution: {integrity: sha512-/wNZ8twnEQQA4HoHu22+vcsdru6pWPWxW+7w+FlxT6Id7PE/WIbZmVKkte+PF72e0F2dnImFeHD2syyE1Mw6MQ==}
'@opentelemetry/core@2.11.0':
resolution: {integrity: sha512-7YP44XH0tV6+Mb54x2YGf84i7yi+31MBZlE8JwvozkxyTvXbSp10X7cI7YE49ChJ3shMJoBmCJF3+1QFBJctGA==}
engines: {node: ^18.19.0 || >=20.6.0}
peerDependencies:
'@opentelemetry/api': '>=1.0.0 <1.10.0'
'@opentelemetry/resources@2.10.0':
resolution: {integrity: sha512-q6MMm2zhggzsHVNbabYwut+a6nbuQQe3URUoxaojM/8K1IBfwwPzvxIjNi2/lI1TFe+fMHMW9MWhrtDLEXEnkA==}
'@opentelemetry/resources@2.11.0':
resolution: {integrity: sha512-Ie7+8q8MDF4FAEQCKVMTx3ReUvxiIAgIiiW3c9JdmP8+HMcDy20puT+AHjexnExgnbvBxjQ9fjkFDWrikJ2jQA==}
engines: {node: ^18.19.0 || >=20.6.0}
peerDependencies:
'@opentelemetry/api': '>=1.3.0 <1.10.0'
'@opentelemetry/sdk-metrics@2.10.0':
resolution: {integrity: sha512-t6r1VSvXNtSDnPXU1FbZeetJb7yyovHmgu0wRSoftxtE0g2rSNhQZQUy69sRUCL+iioJpX8SN/S6wq6ZtvLySQ==}
'@opentelemetry/sdk-metrics@2.11.0':
resolution: {integrity: sha512-7GXXcObyHyDUUSG+L+kJoquty01bzm7ivE7+SSgXXJcHuPzGviptxwARmI2c+bnnxjexGQbJnyNlN8HxBP/Y7A==}
engines: {node: ^18.19.0 || >=20.6.0}
peerDependencies:
'@opentelemetry/api': '>=1.9.0 <1.10.0'
@@ -1480,6 +1480,7 @@ packages:
'@smithy/core@3.24.1':
resolution: {integrity: sha512-3mT7o4qQyUWttYnVK3A0Z/u3Xha3E81tXn32Tz6vjZiUXhBrkEivpw1hBYfh84iFF9CSzkBU9Y1DJ3Q6RQ231g==}
engines: {node: '>=18.0.0'}
deprecated: Deprecated due to bug in browser bundling instructions https://github.com/smithy-lang/smithy-typescript/issues/2025
'@smithy/credential-provider-imds@4.3.1':
resolution: {integrity: sha512-0S/acwHnqX4WrjXzhdiDRxsG2s9SC0cpPIK9nZ1R6UOHd+j7uL28+4bHu22urbLk2TVw3fkp6na/+fkUt/pLNQ==}
@@ -3238,8 +3239,8 @@ packages:
peerDependencies:
typescript: '>=4.2.0'
ts-jest@29.4.9:
resolution: {integrity: sha512-LTb9496gYPMCqjeDLdPrKuXtncudeV1yRZnF4Wo5l3SFi0RYEnYRNgMrFIdg+FHvfzjCyQk1cLncWVqiSX+EvQ==}
ts-jest@29.4.12:
resolution: {integrity: sha512-Ov6ClY53Fflh6BGAnY2DlTq1hYDrTycz2PVTXBWFW2CU+9zrEqAp9fWdGXl42EXO5RLSFAcAZ2JFKbP+zBTFfw==}
engines: {node: ^14.15.0 || ^16.10.0 || ^18.0.0 || >=20.0.0}
hasBin: true
peerDependencies:
@@ -5110,22 +5111,22 @@ snapshots:
'@opentelemetry/api@1.9.1': {}
'@opentelemetry/core@2.10.0(@opentelemetry/api@1.9.1)':
'@opentelemetry/core@2.11.0(@opentelemetry/api@1.9.1)':
dependencies:
'@opentelemetry/api': 1.9.1
'@opentelemetry/semantic-conventions': 1.43.0
'@opentelemetry/resources@2.10.0(@opentelemetry/api@1.9.1)':
'@opentelemetry/resources@2.11.0(@opentelemetry/api@1.9.1)':
dependencies:
'@opentelemetry/api': 1.9.1
'@opentelemetry/core': 2.10.0(@opentelemetry/api@1.9.1)
'@opentelemetry/core': 2.11.0(@opentelemetry/api@1.9.1)
'@opentelemetry/semantic-conventions': 1.43.0
'@opentelemetry/sdk-metrics@2.10.0(@opentelemetry/api@1.9.1)':
'@opentelemetry/sdk-metrics@2.11.0(@opentelemetry/api@1.9.1)':
dependencies:
'@opentelemetry/api': 1.9.1
'@opentelemetry/core': 2.10.0(@opentelemetry/api@1.9.1)
'@opentelemetry/resources': 2.10.0(@opentelemetry/api@1.9.1)
'@opentelemetry/core': 2.11.0(@opentelemetry/api@1.9.1)
'@opentelemetry/resources': 2.11.0(@opentelemetry/api@1.9.1)
'@opentelemetry/semantic-conventions@1.43.0': {}
@@ -5574,7 +5575,7 @@ snapshots:
globby: 11.1.0
is-glob: 4.0.3
minimatch: 9.0.9
semver: 7.8.0
semver: 7.8.5
ts-api-utils: 1.4.3(typescript@5.5.4)
optionalDependencies:
typescript: 5.5.4
@@ -6426,7 +6427,7 @@ snapshots:
'@babel/parser': 7.29.3
'@istanbuljs/schema': 0.1.6
istanbul-lib-coverage: 3.2.2
semver: 7.8.0
semver: 7.8.5
transitivePeerDependencies:
- supports-color
@@ -6705,7 +6706,7 @@ snapshots:
jest-util: 29.7.0
natural-compare: 1.4.0
pretty-format: 29.7.0
semver: 7.8.0
semver: 7.8.5
transitivePeerDependencies:
- supports-color
@@ -6828,7 +6829,7 @@ snapshots:
make-dir@4.0.0:
dependencies:
semver: 7.8.0
semver: 7.8.5
make-error@1.3.6: {}
@@ -7162,8 +7163,7 @@ snapshots:
semver@7.8.0: {}
semver@7.8.5:
optional: true
semver@7.8.5: {}
sharp@0.35.4(@types/node@22.7.4):
dependencies:
@@ -7327,7 +7327,7 @@ snapshots:
dependencies:
typescript: 5.5.4
ts-jest@29.4.9(@babel/core@7.29.7)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.29.7))(jest-util@29.7.0)(jest@29.7.0(@types/node@22.7.4))(typescript@5.5.4):
ts-jest@29.4.12(@babel/core@7.29.7)(@jest/transform@29.7.0)(@jest/types@29.6.3)(babel-jest@29.7.0(@babel/core@7.29.7))(jest-util@29.7.0)(jest@29.7.0(@types/node@22.7.4))(typescript@5.5.4):
dependencies:
bs-logger: 0.2.6
fast-json-stable-stringify: 2.1.0
@@ -7336,7 +7336,7 @@ snapshots:
json5: 2.2.3
lodash.memoize: 4.1.2
make-error: 1.3.6
semver: 7.8.0
semver: 7.8.5
type-fest: 4.41.0
typescript: 5.5.4
yargs-parser: 21.1.1
+1 -1
View File
@@ -127,7 +127,7 @@ impl Job {
}
/// Serialise Arrow batches as a single IPC stream for the TypeScript layer.
pub(crate) fn batches_to_ipc_buffer(batches: &[RecordBatch]) -> napi::Result<Buffer> {
fn batches_to_ipc_buffer(batches: &[RecordBatch]) -> napi::Result<Buffer> {
let Some(first) = batches.first() else {
return Ok(Buffer::from(Vec::<u8>::new()));
};
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "lancedb-python"
version = "0.39.0-beta.3"
version = "0.39.0-beta.4"
publish = false
edition.workspace = true
description = "Python bindings for LanceDB"
+2
View File
@@ -37,6 +37,8 @@ from .functions import (
UdfDefinition as UdfDefinition,
udf as udf,
)
from .secrets import EnvVarSecret as EnvVarSecret
from .secrets import SecretInfo as SecretInfo
from .materialized_view import (
AsyncMaterializedView,
MaterializedView,
+5
View File
@@ -153,6 +153,11 @@ class Connection(object):
async def get_function(self, name: str, version: str) -> str: ...
async def list_functions(self) -> List[str]: ...
async def drop_function(self, name: str, version: str) -> bool: ...
async def create_secret(self, name: str, value: str) -> None: ...
async def alter_secret(self, name: str, value: str) -> None: ...
async def list_secrets(self) -> List[str]: ...
async def drop_secret(self, name: str) -> None: ...
async def describe_secret(self, name: str) -> Dict[str, str]: ...
async def list_jobs(self) -> List[JobInfo]: ...
async def cancel_job(self, job_id: str) -> bool: ...
async def execute_query_async(
+161 -11
View File
@@ -17,6 +17,7 @@ from typing import (
List,
Literal,
Optional,
Sequence,
Union,
)
from uuid import UUID
@@ -57,6 +58,7 @@ from .materialized_view import (
SelectArg,
normalize_select,
)
from .secrets import EnvVarSecret, SecretInfo, validate_secret_name
from .table import (
AsyncTable,
LanceTable,
@@ -692,15 +694,47 @@ class DBConnection(EnforceOverrides):
"""
raise NotImplementedError("serialize is not supported for this connection type")
def create_function(self, definition: UdfDefinition) -> FunctionVersion:
def create_function(
self,
definition: UdfDefinition,
*,
secrets: Optional[Sequence[EnvVarSecret]] = None,
) -> FunctionVersion:
"""Register a scalar Python UDF and wait for its immutable version.
This is the blocking counterpart of :meth:`create_function_async`.
Local connections raise ``NotImplementedError``.
"""
return self.create_function_async(definition).wait()
def create_function_async(self, definition: UdfDefinition) -> Job[FunctionVersion]:
Parameters
----------
definition : UdfDefinition
A callable decorated with [udf][lancedb.udf].
secrets : sequence of EnvVarSecret, optional
One [EnvVarSecret][lancedb.secrets.EnvVarSecret] per credential the
Function needs, each naming a Secret and the environment variable
its value arrives in. The Function's source is unchanged by this;
it reads the variable the way it already did.
Examples
--------
```python
db.create_secret("openai-prod", os.environ["OPENAI_API_KEY"])
db.create_function(
analyze_caption,
secrets=[
EnvVarSecret(secret="openai-prod", env_variable="OPENAI_API_KEY")
],
)
```
"""
return self.create_function_async(definition, secrets=secrets).wait()
def create_function_async(
self,
definition: UdfDefinition,
*,
secrets: Optional[Sequence[EnvVarSecret]] = None,
) -> Job[FunctionVersion]:
"""Register a scalar Python UDF through the remote Function catalog.
Submission returns a typed job. The immutable Function version becomes
@@ -745,6 +779,62 @@ class DBConnection(EnforceOverrides):
"Function catalog operations are not supported for this connection type"
)
def create_secret(self, name: str, value: str) -> None:
"""Create a named Secret in this database.
Fails if the name is taken, so a create never silently becomes a
rotation. Nothing reads the value back: it is bound to a Function by
name and resolved by the service when that Function runs. Local
connections raise ``NotImplementedError``.
"""
raise NotImplementedError(
"Secret operations are not supported for this connection type"
)
def alter_secret(self, name: str, value: str) -> None:
"""Replace the credential behind an existing Secret.
Fails if it does not exist. Every Function bound to the Secret uses the
new value from its next job, and no new Function version is created --
which is how a rotation reaches columns pinned to a version registered
before it. Local connections raise ``NotImplementedError``.
"""
raise NotImplementedError(
"Secret operations are not supported for this connection type"
)
def list_secrets(self) -> List[str]:
"""The names of every Secret in this database.
Names only. No method returns a stored credential, by construction
rather than by policy. Local connections raise ``NotImplementedError``.
"""
raise NotImplementedError(
"Secret operations are not supported for this connection type"
)
def drop_secret(self, name: str) -> None:
"""Drop a Secret.
Functions bound to it fail at their next job, naming the Secret; that
is the revocation path. The name becomes free to reuse, and a new
Secret under it is picked up by everything still bound to that name.
Local connections raise ``NotImplementedError``.
"""
raise NotImplementedError(
"Secret operations are not supported for this connection type"
)
def describe_secret(self, name: str) -> SecretInfo:
"""What this database records about a Secret: name and timestamps.
Never the value -- there is no code path that could return one. Local
connections raise ``NotImplementedError``.
"""
raise NotImplementedError(
"Secret operations are not supported for this connection type"
)
def open_job(self, job_id: str) -> Job:
"""Open a server-side job by id, returning a handle with its record
already populated.
@@ -1457,8 +1547,13 @@ class LanceDBConnection(DBConnection):
return Job(LOOP.run(self._conn.open_job(job_id)))
@override
def create_function_async(self, definition: UdfDefinition) -> Job[FunctionVersion]:
job = LOOP.run(self._conn.create_function_async(definition))
def create_function_async(
self,
definition: UdfDefinition,
*,
secrets: Optional[Sequence[EnvVarSecret]] = None,
) -> Job[FunctionVersion]:
job = LOOP.run(self._conn.create_function_async(definition, secrets=secrets))
return Job(job)
@override
@@ -1473,6 +1568,26 @@ class LanceDBConnection(DBConnection):
def drop_function(self, name: str, *, version: str) -> bool:
return LOOP.run(self._conn.drop_function(name, version=version))
@override
def create_secret(self, name: str, value: str) -> None:
LOOP.run(self._conn.create_secret(name, value))
@override
def alter_secret(self, name: str, value: str) -> None:
LOOP.run(self._conn.alter_secret(name, value))
@override
def list_secrets(self) -> List[str]:
return LOOP.run(self._conn.list_secrets())
@override
def drop_secret(self, name: str) -> None:
LOOP.run(self._conn.drop_secret(name))
@override
def describe_secret(self, name: str) -> SecretInfo:
return LOOP.run(self._conn.describe_secret(name))
@override
def list_jobs(self) -> List[JobInfo]:
"""List server-side jobs across the database's tables."""
@@ -2265,18 +2380,23 @@ class AsyncConnection(object):
return AsyncJob(await self._inner.open_job(job_id))
async def create_function_async(
self, definition: UdfDefinition
self,
definition: UdfDefinition,
*,
secrets: Optional[Sequence[EnvVarSecret]] = None,
) -> AsyncJob[FunctionVersion]:
"""Register a scalar Python UDF through the remote Function catalog.
The returned typed job resolves to the immutable Function version.
Local connections raise ``NotImplementedError``.
``secrets`` is a sequence of
[EnvVarSecret][lancedb.secrets.EnvVarSecret], each naming a Secret and
the environment variable its value arrives in. Local connections raise
``NotImplementedError``.
"""
if not isinstance(definition, UdfDefinition):
raise TypeError("create_function_async requires a @udf definition")
inner = await self._inner.create_function_async(
definition.registration_request.to_canonical_json()
)
request = definition.bind_secrets(secrets)
inner = await self._inner.create_function_async(request.to_canonical_json())
return _typed_job(inner, FunctionVersion.from_json)
async def get_function(self, name: str, *, version: str) -> FunctionVersion:
@@ -2298,6 +2418,36 @@ class AsyncConnection(object):
"""Drop one exact immutable Function version from the remote catalog."""
return await self._inner.drop_function(name, version)
async def create_secret(self, name: str, value: str) -> None:
"""Create a named Secret in this database.
Fails if the name is taken, so a create never silently becomes a
rotation. Nothing reads the value back.
"""
await self._inner.create_secret(validate_secret_name(name), value)
async def alter_secret(self, name: str, value: str) -> None:
"""Replace the credential behind an existing Secret.
Fails if it does not exist. Bound Functions use the new value from
their next job, with no new Function version.
"""
await self._inner.alter_secret(validate_secret_name(name), value)
async def list_secrets(self) -> List[str]:
"""The names of every Secret in this database. Names only."""
return await self._inner.list_secrets()
async def drop_secret(self, name: str) -> None:
"""Drop a Secret. Bound Functions fail at their next job."""
await self._inner.drop_secret(validate_secret_name(name))
async def describe_secret(self, name: str) -> SecretInfo:
"""What this database records about a Secret. Never the value."""
return SecretInfo.from_json(
await self._inner.describe_secret(validate_secret_name(name))
)
async def list_jobs(self) -> List[JobInfo]:
"""List server-side jobs across the database's tables."""
return await self._inner.list_jobs()
+64 -5
View File
@@ -4,7 +4,7 @@
"""Canonical Function values exchanged with LanceDB Enterprise services.
These immutable models contain client/wire state only. Catalog persistence,
environment bake, and execution are owned by Sophon.
environment bake, secret resolution, and execution are owned by Sophon.
``RefreshColumnResult`` is also the backend-neutral result of a local
expression-backed refresh job.
"""
@@ -25,7 +25,7 @@ import re
import sys
import textwrap
import types
from collections.abc import Mapping
from collections.abc import Mapping, Sequence
from datetime import date, datetime
from typing import (
Annotated,
@@ -50,6 +50,7 @@ from pydantic import (
)
from .schema import is_blob_v2_field as _is_blob_v2_field
from .secrets import EnvVarSecret
_Int32 = conint(strict=True, ge=-(2**31), le=2**31 - 1)
_UInt32 = conint(strict=True, ge=0, le=2**32 - 1)
@@ -309,6 +310,7 @@ class FunctionVersion(_RemoteValue):
runtime: PythonRuntimeSpec
runtime_digest: str
environment_digest: str
secret_env_bindings: Mapping[str, str] = {}
created_at: str
def __call__(self, **inputs: Any) -> FunctionApplication:
@@ -370,12 +372,18 @@ class FunctionVersion(_RemoteValue):
class FunctionRegistrationRequest(_RemoteValue):
"""Stable remote registration envelope produced by :func:`udf`."""
"""Stable remote registration envelope produced by :func:`udf`.
Credential values deliberately have no field here. The only secret-shaped
thing a client sends is ``secret_env_bindings``: the name of a Secret the
database already holds, which the remote service resolves at execution.
"""
name: str
artifact: FunctionArtifactRequest
signature: FunctionSignature
runtime: PythonRuntimeSpec
secret_env_bindings: Mapping[str, str] = {}
class FunctionVersionRef(_OpenRemoteValue):
@@ -524,6 +532,8 @@ class RefreshColumnResult(_RemoteValue):
_FUNCTION_NAME = re.compile(r"^[A-Za-z_][A-Za-z0-9_.-]*$")
_DECLARED_SECRET = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
_FUNCTION_BLOB_V2_TYPE = "blob_v2"
_ARROW_EXTENSION_NAME_KEY = "ARROW:extension:name"
_BLOB_V2_EXTENSION_NAME = "lance.blob.v2"
@@ -1265,9 +1275,56 @@ class UdfDefinition:
@property
def registration_request(self) -> FunctionRegistrationRequest:
"""The immutable request sent by ``create_function_async``."""
"""The immutable request sent by ``create_function_async``.
Carries no secret bindings. Binding is a registration-time decision,
so a Function bound to Secrets is registered through :meth:`bind_secrets`,
which is what ``create_function`` calls.
"""
return self._request
def bind_secrets(
self, secrets: Optional[Sequence[EnvVarSecret]]
) -> FunctionRegistrationRequest:
"""The registration request for this definition bound to ``secrets``.
Binding does not change the Function's source: each
[EnvVarSecret][lancedb.secrets.EnvVarSecret] names a Secret and the
environment variable its value should arrive in, and the Function reads
that variable the way it already did. Whether the named Secrets exist is
the server's answer, not this one.
"""
bindings = () if secrets is None else tuple(secrets)
wrong_type = [
binding for binding in bindings if not isinstance(binding, EnvVarSecret)
]
if wrong_type:
kinds = sorted({type(binding).__name__ for binding in wrong_type})
raise TypeError(
f"Function secrets must be EnvVarSecret values, not {kinds!r}; a "
"credential value is never sent to this API"
)
variables = [binding.env_variable for binding in bindings]
duplicates = sorted({name for name in variables if variables.count(name) > 1})
if duplicates:
raise ValueError(
"a Function binds each environment variable once; duplicated: "
f"{duplicates!r}"
)
# `env` is ordinary configuration carried in the definition, so a name in
# both would have a value visible in the Function's record and a value
# that is not. Refuse rather than pick.
environment = self._request.runtime.env or {}
overlap = sorted(set(environment) & set(variables))
if overlap:
raise ValueError(
f"Function env and secret bindings must be disjoint: {overlap!r}"
)
if not bindings:
return self._request
resolved = {binding.env_variable: binding.secret for binding in bindings}
return self._request._copy(update={"secret_env_bindings": resolved})
def __call__(self, *args, **kwargs):
return self._function(*args, **kwargs)
@@ -1332,7 +1389,9 @@ def udf(
conda_channels : sequence of str, optional
Conda channels in priority order; requires ``conda``.
env : mapping of str to str, optional
Environment variables included in the Function definition.
Environment variables included in the Function definition. Not for
credentials -- these are ordinary configuration, stored with the
Function and visible wherever it is.
python_version : str, optional
Remote Python major/minor version. Defaults to the client version.
gpu : bool, default False
+39 -3
View File
@@ -7,7 +7,16 @@ import json
import logging
from concurrent.futures import ThreadPoolExecutor
import sys
from typing import TYPE_CHECKING, Any, Dict, Iterable, List, Optional, Union
from typing import (
TYPE_CHECKING,
Any,
Dict,
Iterable,
List,
Optional,
Sequence,
Union,
)
from urllib.parse import urlparse
from uuid import UUID
import warnings
@@ -29,6 +38,7 @@ from ..job import AsyncJob, Job
from ..sql import Query as SqlQuery
from ..sql import QueryDescription
from ..materialized_view import MaterializedView, SelectArg
from ..secrets import EnvVarSecret, SecretInfo
if TYPE_CHECKING:
from .._lancedb import JobInfo
@@ -746,8 +756,14 @@ class RemoteDBConnection(DBConnection):
return Job(LOOP.run(self._conn.open_job(job_id)))
@override
def create_function_async(self, definition: UdfDefinition) -> Job[FunctionVersion]:
return Job(LOOP.run(self._conn.create_function_async(definition)))
def create_function_async(
self,
definition: UdfDefinition,
*,
secrets: Optional[Sequence[EnvVarSecret]] = None,
) -> Job[FunctionVersion]:
job = LOOP.run(self._conn.create_function_async(definition, secrets=secrets))
return Job(job)
@override
def get_function(self, name: str, *, version: str) -> FunctionVersion:
@@ -761,6 +777,26 @@ class RemoteDBConnection(DBConnection):
def drop_function(self, name: str, *, version: str) -> bool:
return LOOP.run(self._conn.drop_function(name, version=version))
@override
def create_secret(self, name: str, value: str) -> None:
LOOP.run(self._conn.create_secret(name, value))
@override
def alter_secret(self, name: str, value: str) -> None:
LOOP.run(self._conn.alter_secret(name, value))
@override
def describe_secret(self, name: str) -> SecretInfo:
return LOOP.run(self._conn.describe_secret(name))
@override
def list_secrets(self) -> List[str]:
return LOOP.run(self._conn.list_secrets())
@override
def drop_secret(self, name: str) -> None:
LOOP.run(self._conn.drop_secret(name))
@override
def list_jobs(self) -> List["JobInfo"]:
"""List server-side jobs across the database's tables."""
+169
View File
@@ -0,0 +1,169 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright The LanceDB Authors
"""Named Secrets, and the bindings that deliver them to Functions.
A Secret is a database-scoped named credential. Nothing in this module holds a
value: :class:`EnvVarSecret` names one and says which environment variable it
should arrive in, and the value is resolved by the remote service when a
Function bound to it runs. No API returns a stored credential, by construction
rather than by policy -- there is no code path that could.
"""
from __future__ import annotations
import re
# The same characters LanceDB already admits in a namespace or table name, and
# no positional rule on top of them: a segment may begin with `_`, `-` or `.`
# today, so anything narrower would put Secrets out of reach inside namespaces
# that already exist. Matches the service, which admits the same set.
_SECRET_NAME = re.compile(r"^[A-Za-z0-9_.-]{1,255}$")
_ENV_VARIABLE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
def validate_secret_name(name: str) -> str:
"""Check a Secret name locally and return it unchanged."""
if not isinstance(name, str):
raise TypeError(f"Secret name must be a string, not {type(name).__name__}")
if not _SECRET_NAME.fullmatch(name):
raise ValueError(f"invalid Secret name: {name!r}")
return name
def validate_env_variable(name: str) -> str:
"""Check an environment variable name locally and return it unchanged."""
if not isinstance(name, str):
raise TypeError(
f"environment variable name must be a string, not {type(name).__name__}"
)
if not _ENV_VARIABLE.fullmatch(name):
raise ValueError(f"invalid environment variable name: {name!r}")
return name
class EnvVarSecret:
"""A Secret bound to the environment variable a Function's library reads.
Pass these in the ``secrets`` sequence of
[DBConnection.create_function][lancedb.db.DBConnection.create_function]. The
Function's source is unchanged by binding: it reads ``OPENAI_API_KEY`` the
way it always did, and the binding is what puts a value there.
This is a local value. Constructing it contacts no server, so it always
succeeds and says nothing about whether the Secret exists; that is checked
at registration, where a mistyped Secret name surfaces as a clear "does not
exist" naming both the Secret and the variable bound to it. A mistyped
*variable* name cannot be caught anywhere -- nothing knows which variables a
Function reads -- so it surfaces on the first rows instead.
The type exists so a credential cannot be passed by accident. A bare string
in the same position is a plausible-looking mistake with the opposite
meaning, and it reads identically in a diff.
Parameters
----------
secret : str
The Secret's database-scoped name.
env_variable : str
The environment variable the Function reads it from.
Examples
--------
>>> from lancedb import EnvVarSecret
>>> binding = EnvVarSecret(secret="openai-prod", env_variable="OPENAI_API_KEY")
>>> binding.secret, binding.env_variable
('openai-prod', 'OPENAI_API_KEY')
"""
__slots__ = ("_secret", "_env_variable")
def __init__(self, secret: str, env_variable: str):
self._secret = validate_secret_name(secret)
self._env_variable = validate_env_variable(env_variable)
@property
def secret(self) -> str:
"""The Secret's database-scoped name."""
return self._secret
@property
def env_variable(self) -> str:
"""The environment variable the value is delivered in."""
return self._env_variable
def __repr__(self) -> str:
return (
f"EnvVarSecret(secret={self._secret!r}, "
f"env_variable={self._env_variable!r})"
)
def __eq__(self, other: object) -> bool:
return (
isinstance(other, EnvVarSecret)
and other._secret == self._secret
and other._env_variable == self._env_variable
)
def __hash__(self) -> int:
return hash((EnvVarSecret, self._secret, self._env_variable))
class SecretInfo:
"""What a database records about a Secret. Never its value.
Returned by
[DBConnection.describe_secret][lancedb.db.DBConnection.describe_secret].
"""
__slots__ = ("_name", "_created_at", "_updated_at")
def __init__(self, name: str, created_at: str, updated_at: str):
self._name = name
self._created_at = created_at
self._updated_at = updated_at
@property
def name(self) -> str:
"""The Secret's database-scoped name."""
return self._name
@property
def created_at(self) -> str:
"""When the Secret was created, as an RFC 3339 timestamp."""
return self._created_at
@property
def updated_at(self) -> str:
"""When the Secret's value was last rotated, as an RFC 3339 timestamp."""
return self._updated_at
@classmethod
def from_json(cls, value: dict) -> "SecretInfo":
return cls(
name=value["name"],
created_at=value["created_at"],
updated_at=value["updated_at"],
)
def __repr__(self) -> str:
return (
f"SecretInfo(name={self._name!r}, created_at={self._created_at!r}, "
f"updated_at={self._updated_at!r})"
)
def __eq__(self, other: object) -> bool:
return (
isinstance(other, SecretInfo)
and other._name == self._name
and other._created_at == self._created_at
and other._updated_at == self._updated_at
)
__all__ = [
"EnvVarSecret",
"SecretInfo",
"validate_env_variable",
"validate_secret_name",
]
@@ -37,6 +37,22 @@ def job_result(name: str) -> dict:
return json.loads(fixture(name))["result"]
def assert_no_secret_values(value):
"""No client value models a resolved credential, at any nesting depth."""
if isinstance(value, dict):
for key, child in value.items():
assert key not in {
"secret_value",
"secret_values",
"resolved_secret",
"resolved_secrets",
}
assert_no_secret_values(child)
elif isinstance(value, list):
for child in value:
assert_no_secret_values(child)
def test_public_function_values_are_in_api_reference():
docs = Path(__file__).parents[3] / "docs" / "src" / "python" / "python.md"
rendered = docs.read_text()
@@ -94,6 +110,7 @@ def test_function_version_identity_is_immutable_and_exact():
version = FunctionVersion.from_json(json.dumps(value))
assert version.name == "embed"
assert version.version == "fv_01K3EXACT"
assert dict(version.secret_env_bindings) == {"HF_TOKEN": "hf-prod"}
with pytest.raises((TypeError, ValueError)):
version.version = "fv_changed"
@@ -276,6 +293,25 @@ def test_refresh_result_rejects_non_u64_values(field):
RefreshColumnResult.from_json(json.dumps(value))
def test_canonical_client_values_carry_bindings_and_no_credentials():
"""A binding names a Secret; the credential behind it has no client field."""
version = FunctionVersion.from_json(
json.dumps(job_result("remote_function_job.json"))
)
canonical = json.loads(version.to_canonical_json())
assert canonical["secret_env_bindings"] == {"HF_TOKEN": "hf-prod"}
assert_no_secret_values(canonical)
def test_a_version_without_bindings_keeps_the_original_wire_shape():
"""Every Function registered before Secrets existed serializes unchanged."""
value = job_result("remote_function_job.json")
del value["secret_env_bindings"]
version = FunctionVersion.from_json(json.dumps(value))
assert dict(version.secret_env_bindings) == {}
assert "secret_env_bindings" not in json.loads(version.to_canonical_json())
class _FunctionDeclarationInner:
def __init__(self):
self.calls = []
@@ -11,6 +11,7 @@ import types
from datetime import date
import http.server
import json
import os
from pathlib import Path
import subprocess
import sys
@@ -21,6 +22,7 @@ import pyarrow as pa
import pytest
import lancedb
from lancedb.background_loop import LOOP
from lancedb.functions import (
PythonRuntimeSpec,
UdfDefinition,
@@ -28,6 +30,7 @@ from lancedb.functions import (
_GRAMMAR_PRIMITIVES,
udf,
)
from lancedb.secrets import EnvVarSecret
THRESHOLD = 20
_CACHE = None
@@ -53,6 +56,15 @@ def normalize_score(value: float) -> float:
return value / 100.0
@udf(
pip=["openai==3.7.0"],
env={"MODE": "test"},
python_version="3.12",
)
def analyze_caption(caption: str) -> str:
return caption.strip()
def test_scalar_udf_matches_shared_registration_golden_and_remains_callable():
assert isinstance(normalize_score, UdfDefinition)
assert normalize_score(25.0) == 0.25
@@ -69,6 +81,232 @@ def test_scalar_udf_matches_shared_registration_golden_and_remains_callable():
}
def test_secret_bound_udf_matches_its_shared_registration_golden():
assert analyze_caption(" hello ") == "hello"
bound = analyze_caption.bind_secrets(
[EnvVarSecret(secret="openai-prod", env_variable="OPENAI_API_KEY")]
)
assert (
bound.to_canonical_json()
== (FIXTURES / "remote_function_secret_registration_request.canonical.json")
.read_text()
.strip()
)
def test_an_unbound_request_carries_no_binding_at_all():
"""Binding is a registration-time decision, so the definition holds none.
The decorator declares nothing about secrets, which is what makes the PRD's
claim true: a Function's source and its registration request are identical
whether or not a credential is later bound to it.
"""
unbound = json.loads(analyze_caption.registration_request.to_canonical_json())
assert "secret_env_bindings" not in unbound
assert "OPENAI_API_KEY" not in json.dumps(unbound)
def test_binding_a_secret_leaves_the_packaged_artifact_untouched():
"""The artifact is source bytes and nothing else, with or without secrets."""
bound = analyze_caption.bind_secrets(
[EnvVarSecret(secret="openai-prod", env_variable="OPENAI_API_KEY")]
)
assert bound.artifact == analyze_caption.registration_request.artifact
assert bound.artifact.digest == analyze_caption.registration_request.artifact.digest
def test_a_function_declaring_no_secret_is_registered_exactly_as_before():
"""The compatibility claim: nothing about the no-secret path moves."""
assert (
normalize_score.bind_secrets(None).to_canonical_json()
== normalize_score.registration_request.to_canonical_json()
)
assert (
"secret_env_bindings"
not in normalize_score.registration_request.to_canonical_json()
)
def test_a_function_binds_each_variable_once():
with pytest.raises(ValueError, match="binds each environment variable once"):
analyze_caption.bind_secrets(
[
EnvVarSecret(secret="openai-prod", env_variable="OPENAI_API_KEY"),
EnvVarSecret(secret="openai-staging", env_variable="OPENAI_API_KEY"),
]
)
def test_bindings_may_not_collide_with_plain_configuration():
"""`env` is stored with the Function; a Secret is not. Refuse, do not pick."""
with pytest.raises(ValueError, match="must be disjoint"):
analyze_caption.bind_secrets(
[EnvVarSecret(secret="mode-prod", env_variable="MODE")]
)
def test_a_function_binds_at_most_sixteen_secrets():
"""The cap lives in Rust, so no language surface can be talked past it.
Registering through the typed API and hand-rolling the request envelope
reach the same boundary, and neither reaches the wire.
"""
bindings = [
EnvVarSecret(secret=f"secret-{index}", env_variable=f"TOKEN_{index}")
for index in range(17)
]
with _mock_remote_function_catalog() as (host, state):
db = lancedb.connect(
"db://dev",
api_key="fake",
host_override=host,
client_config={"retry_config": {"retries": 0}},
)
with pytest.raises(ValueError, match="at most 16 secrets"):
db.create_function(normalize_score, secrets=bindings)
envelope = json.loads(normalize_score.registration_request.to_canonical_json())
envelope["secret_env_bindings"] = {
f"TOKEN_{index}": f"secret-{index}" for index in range(17)
}
async def submit_envelope():
return await db._conn._inner.create_function_async(json.dumps(envelope))
with pytest.raises(ValueError, match="at most 16 secrets"):
LOOP.run(submit_envelope())
assert state["requests"] == []
def test_binding_names_are_validated_below_the_python_api():
"""The low-level entry point reaches the same validator the typed API does.
Registration envelopes can be hand-rolled past ``bind_secrets``, so the
grammar and the disjointness rule live in Rust, above the backend.
"""
with _mock_remote_function_catalog() as (host, state):
db = lancedb.connect(
"db://dev",
api_key="fake",
host_override=host,
client_config={"retry_config": {"retries": 0}},
)
envelope = json.loads(analyze_caption.registration_request.to_canonical_json())
envelope["secret_env_bindings"] = {
"BAD=NAME": "openai-prod",
"TOKEN_0": "secret-0",
}
envelope["runtime"]["env"]["TOKEN_0"] = "public"
async def submit_envelope():
return await db._conn._inner.create_function_async(json.dumps(envelope))
with pytest.raises(ValueError, match="portable"):
LOOP.run(submit_envelope())
assert state["requests"] == []
_SECRET_DEBUG_LOG_SOURCE = """
import http.server
import json
import threading
import lancedb
class Handler(http.server.BaseHTTPRequestHandler):
def log_message(self, *args):
pass
def do_POST(self):
self.rfile.read(int(self.headers.get("Content-Length", "0")))
payload = json.dumps({}).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
server = http.server.ThreadingHTTPServer(("localhost", 0), Handler)
threading.Thread(target=server.serve_forever, daemon=True).start()
try:
db = lancedb.connect(
"db://dev",
api_key="API_KEY_SENTINEL",
host_override="http://localhost:%d" % server.server_address[1],
client_config={"retry_config": {"retries": 0}},
)
db.create_secret("openai-prod", "SECRET_VALUE_SENTINEL")
finally:
server.shutdown()
"""
def test_a_credential_never_reaches_a_debug_log(tmp_path):
"""The logger sees the serialized body, so no value-side redaction reaches it.
Runs in a subprocess because the Rust logger reads ``LANCEDB_LOG`` once, at
import.
"""
script = tmp_path / "write_secret.py"
script.write_text(_SECRET_DEBUG_LOG_SOURCE)
result = subprocess.run(
[sys.executable, str(script)],
check=True,
capture_output=True,
text=True,
env={**os.environ, "LANCEDB_LOG": "debug"},
)
output = result.stdout + result.stderr
# Without this the test passes when debug logging is simply off.
assert "Sending request_id=" in output, output
assert "SECRET_VALUE_SENTINEL" not in output
assert "API_KEY_SENTINEL" not in output
def test_a_credential_value_is_rejected_in_the_binding_position():
"""The one mistake the typed binding exists to stop."""
with pytest.raises(TypeError, match="EnvVarSecret"):
analyze_caption.bind_secrets(["sk-live-0001"])
@pytest.mark.parametrize(
("secret", "variable", "message"),
[
("openai-prod", "not-a-var", "invalid environment variable name"),
("openai-prod", "API-TOKEN", "invalid environment variable name"),
("not a name", "API_TOKEN", "invalid Secret name"),
("openai$prod", "API_TOKEN", "invalid Secret name"),
],
)
def test_a_binding_validates_both_names_locally(secret, variable, message):
with pytest.raises(ValueError, match=message):
EnvVarSecret(secret=secret, env_variable=variable)
def test_a_secret_name_admits_what_a_namespace_name_does():
"""A Secret has to be nameable wherever a namespace already is.
LanceDB namespace and table names are `[A-Za-z0-9_.-]` with no rule about
which character comes first, so a name may lead with `_`, `-` or `.`.
Anything narrower here would leave Secrets unaddressable inside namespaces
that already exist -- the reason periods are admitted is the reason the
edges are too.
"""
for name in ["openai.prod.v1", ".hidden", "_internal", "-lead", "trailing."]:
binding = EnvVarSecret(secret=name, env_variable="OPENAI_API_KEY")
assert binding.secret == name
for name in ["", "with/slash", "with$delimiter", "a" * 256]:
with pytest.raises(ValueError, match="invalid Secret name"):
EnvVarSecret(secret=name, env_variable="OPENAI_API_KEY")
def _main_udf_source(
*, threshold: int = 20, input_annotation: str = "int", comparison: str = ">="
) -> str:
@@ -1230,6 +1468,7 @@ def _mock_remote_function_catalog():
"runtime": body["runtime"],
"runtime_digest": "sha256:runtime",
"environment_digest": "sha256:environment",
"secret_env_bindings": body.get("secret_env_bindings", {}),
"created_at": "2026-08-21T00:00:00Z",
}
response = {"job_id": "job-register"}
@@ -1270,6 +1509,21 @@ def _mock_remote_function_catalog():
"version": "fv_exact",
}
response = {"dropped": True}
elif self.path in ("/v1/secrets/create", "/v1/secrets/alter"):
assert set(body) == {"name", "value"}
response = {}
elif self.path == "/v1/secrets/list":
if "page_token" not in body:
response = {
"secrets": [{"name": "openai-prod"}],
"page_token": "next",
}
else:
assert body["page_token"] == "next"
response = {"secrets": [{"name": "hf-prod"}]}
elif self.path == "/v1/secrets/drop":
assert body == {"name": "openai-prod"}
response = {}
else:
status = 404
response = {"error": "not found"}
@@ -1312,6 +1566,71 @@ def test_remote_registration_job_and_exact_version_reopen_round_trip():
)
def test_remote_registration_sends_bindings_and_never_a_credential():
with _mock_remote_function_catalog() as (host, state):
db = lancedb.connect(
"db://dev",
api_key="fake",
host_override=host,
client_config={"retry_config": {"retries": 0}},
)
created = db.create_function(
analyze_caption,
secrets=[EnvVarSecret(secret="openai-prod", env_variable="OPENAI_API_KEY")],
)
assert dict(created.secret_env_bindings) == {"OPENAI_API_KEY": "openai-prod"}
path, create_request = state["requests"][0]
assert path == "/v1/functions/create"
assert create_request["secret_env_bindings"] == {"OPENAI_API_KEY": "openai-prod"}
# The request names a Secret and carries nothing that could be one.
assert create_request == json.loads(
analyze_caption.bind_secrets(
[EnvVarSecret(secret="openai-prod", env_variable="OPENAI_API_KEY")]
).to_canonical_json()
)
def test_remote_secret_verbs_round_trip():
with _mock_remote_function_catalog() as (host, state):
db = lancedb.connect(
"db://dev",
api_key="fake",
host_override=host,
client_config={"retry_config": {"retries": 0}},
)
assert db.create_secret("openai-prod", "sk-live-0001") is None
assert db.alter_secret("openai-prod", "sk-live-0002") is None
assert db.list_secrets() == ["openai-prod", "hf-prod"]
assert db.drop_secret("openai-prod") is None
routes = [path for path, _ in state["requests"]]
assert routes == [
"/v1/secrets/create",
"/v1/secrets/alter",
"/v1/secrets/list",
"/v1/secrets/list",
"/v1/secrets/drop",
]
assert state["requests"][0][1] == {"name": "openai-prod", "value": "sk-live-0001"}
# The listing returns names, and the client has no way to ask for more.
assert state["requests"][2][1] == {}
def test_building_a_binding_contacts_no_server():
"""A binding is a local value: it says nothing about whether the Secret exists.
Existence is the server's answer at registration, where a mistyped name is a
clear error rather than a client-side check that was already stale.
"""
with _mock_remote_function_catalog() as (_host, state):
binding = EnvVarSecret(secret="openai-prod", env_variable="OPENAI_API_KEY")
assert binding.secret == "openai-prod"
assert binding.env_variable == "OPENAI_API_KEY"
assert state["requests"] == []
def test_blocking_remote_registration_returns_function_version():
with _mock_remote_function_catalog() as (host, state):
db = lancedb.connect(
+50
View File
@@ -704,6 +704,56 @@ impl Connection {
})
}
pub fn create_secret(
self_: PyRef<'_, Self>,
name: String,
value: String,
) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
future_into_py(self_.py(), async move {
inner.create_secret(name, value).await.infer_error()
})
}
pub fn alter_secret(
self_: PyRef<'_, Self>,
name: String,
value: String,
) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
future_into_py(self_.py(), async move {
inner.alter_secret(name, value).await.infer_error()
})
}
pub fn list_secrets(self_: PyRef<'_, Self>) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
future_into_py(self_.py(), async move {
inner.list_secrets().await.infer_error()
})
}
pub fn drop_secret(self_: PyRef<'_, Self>, name: String) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
future_into_py(self_.py(), async move {
inner.drop_secret(name).await.infer_error()
})
}
/// Name and timestamps as a plain mapping. `SecretInfo` carries no value,
/// so there is none to filter out here.
pub fn describe_secret(self_: PyRef<'_, Self>, name: String) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
future_into_py(self_.py(), async move {
let info = inner.describe_secret(name).await.infer_error()?;
Ok(HashMap::from([
("name".to_string(), info.name),
("created_at".to_string(), info.created_at),
("updated_at".to_string(), info.updated_at),
]))
})
}
pub fn list_jobs(self_: PyRef<'_, Self>) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
future_into_py(self_.py(), async move {
+98 -95
View File
@@ -10,6 +10,9 @@ resolution-markers = [
"python_full_version < '3.11'",
]
[options]
prerelease-mode = "allow"
[[package]]
name = "accelerate"
version = "1.14.0"
@@ -799,7 +802,7 @@ name = "cuda-bindings"
version = "13.3.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cuda-pathfinder", marker = "(python_full_version < '3.14' and sys_platform == 'emscripten') or (python_full_version < '3.14' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" },
{ name = "cuda-pathfinder" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/a9/21/8464d133752951c154feafb3b65c297e7d80f301183d220bec4c830f1441/cuda_bindings-13.3.1-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:120fcc53d57903df529c3486962c56528cba5b7d6c57c99537320ed9922c8b86", size = 6073403, upload-time = "2026-05-29T23:11:36.22Z" },
@@ -834,37 +837,37 @@ wheels = [
[package.optional-dependencies]
cublas = [
{ name = "nvidia-cublas", marker = "(python_full_version < '3.14' and sys_platform == 'win32') or sys_platform == 'linux'" },
{ name = "nvidia-cublas" },
]
cudart = [
{ name = "nvidia-cuda-runtime", marker = "(python_full_version < '3.14' and sys_platform == 'win32') or sys_platform == 'linux'" },
{ name = "nvidia-cuda-runtime" },
]
cufft = [
{ name = "nvidia-cufft", marker = "(python_full_version < '3.14' and sys_platform == 'win32') or sys_platform == 'linux'" },
{ name = "nvidia-cufft" },
]
cufile = [
{ name = "nvidia-cufile", marker = "sys_platform == 'linux'" },
{ name = "nvidia-cufile" },
]
cupti = [
{ name = "nvidia-cuda-cupti", marker = "(python_full_version < '3.14' and sys_platform == 'win32') or sys_platform == 'linux'" },
{ name = "nvidia-cuda-cupti" },
]
curand = [
{ name = "nvidia-curand", marker = "(python_full_version < '3.14' and sys_platform == 'win32') or sys_platform == 'linux'" },
{ name = "nvidia-curand" },
]
cusolver = [
{ name = "nvidia-cusolver", marker = "(python_full_version < '3.14' and sys_platform == 'win32') or sys_platform == 'linux'" },
{ name = "nvidia-cusolver" },
]
cusparse = [
{ name = "nvidia-cusparse", marker = "(python_full_version < '3.14' and sys_platform == 'win32') or sys_platform == 'linux'" },
{ name = "nvidia-cusparse" },
]
nvjitlink = [
{ name = "nvidia-nvjitlink", marker = "(python_full_version < '3.14' and sys_platform == 'win32') or sys_platform == 'linux'" },
{ name = "nvidia-nvjitlink" },
]
nvrtc = [
{ name = "nvidia-cuda-nvrtc", marker = "(python_full_version < '3.14' and sys_platform == 'win32') or sys_platform == 'linux'" },
{ name = "nvidia-cuda-nvrtc" },
]
nvtx = [
{ name = "nvidia-nvtx", marker = "(python_full_version < '3.14' and sys_platform == 'win32') or sys_platform == 'linux'" },
{ name = "nvidia-nvtx" },
]
[[package]]
@@ -1023,7 +1026,7 @@ name = "exceptiongroup"
version = "1.3.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "typing-extensions", marker = "python_full_version < '3.11'" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" }
wheels = [
@@ -1440,16 +1443,16 @@ resolution-markers = [
"python_full_version < '3.11'",
]
dependencies = [
{ name = "cachetools", marker = "python_full_version < '3.11'" },
{ name = "certifi", marker = "python_full_version < '3.11'" },
{ name = "httpx", marker = "python_full_version < '3.11'" },
{ name = "ibm-cos-sdk", marker = "python_full_version < '3.11'" },
{ name = "lomond", marker = "python_full_version < '3.11'" },
{ name = "packaging", marker = "python_full_version < '3.11'" },
{ name = "pandas", version = "2.2.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
{ name = "requests", marker = "python_full_version < '3.11'" },
{ name = "tabulate", marker = "python_full_version < '3.11'" },
{ name = "urllib3", marker = "python_full_version < '3.11'" },
{ name = "cachetools" },
{ name = "certifi" },
{ name = "httpx" },
{ name = "ibm-cos-sdk" },
{ name = "lomond" },
{ name = "packaging" },
{ name = "pandas", version = "2.2.3", source = { registry = "https://pypi.org/simple" } },
{ name = "requests" },
{ name = "tabulate" },
{ name = "urllib3" },
]
sdist = { url = "https://files.pythonhosted.org/packages/c7/56/2e3df38a1f13062095d7bde23c87a92f3898982993a15186b1bfecbd206f/ibm_watsonx_ai-1.3.42.tar.gz", hash = "sha256:ee5be59009004245d957ce97d1227355516df95a2640189749487614fef674ff", size = 688651, upload-time = "2025-10-01T13:35:41.527Z" }
wheels = [
@@ -1468,17 +1471,17 @@ resolution-markers = [
"python_full_version == '3.11.*'",
]
dependencies = [
{ name = "cachetools", marker = "python_full_version >= '3.11'" },
{ name = "certifi", marker = "python_full_version >= '3.11'" },
{ name = "httpx", marker = "python_full_version >= '3.11'" },
{ name = "ibm-cos-sdk", marker = "python_full_version >= '3.11'" },
{ name = "lomond", marker = "python_full_version >= '3.11'" },
{ name = "packaging", marker = "python_full_version >= '3.11'" },
{ name = "pandas", version = "2.3.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11' and python_full_version < '3.14'" },
{ name = "cachetools" },
{ name = "certifi" },
{ name = "httpx" },
{ name = "ibm-cos-sdk" },
{ name = "lomond" },
{ name = "packaging" },
{ name = "pandas", version = "2.3.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.14'" },
{ name = "pandas", version = "3.0.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.14'" },
{ name = "requests", marker = "python_full_version >= '3.11'" },
{ name = "tabulate", marker = "python_full_version >= '3.11'" },
{ name = "urllib3", marker = "python_full_version >= '3.11'" },
{ name = "requests" },
{ name = "tabulate" },
{ name = "urllib3" },
]
sdist = { url = "https://files.pythonhosted.org/packages/29/a3/c756b534696ab2f3f29882fdb7ca7198b7a5c94e10c0a3a327853d6d6b79/ibm_watsonx_ai-1.5.14.tar.gz", hash = "sha256:a756488bd57e87c0fc51be42dcba871143cfe0ac1e805c497c5047e1e4f13e9d", size = 735804, upload-time = "2026-06-22T12:32:43.85Z" }
wheels = [
@@ -1554,17 +1557,17 @@ resolution-markers = [
"python_full_version < '3.11'",
]
dependencies = [
{ name = "colorama", marker = "python_full_version < '3.11' and sys_platform == 'win32'" },
{ name = "decorator", marker = "python_full_version < '3.11'" },
{ name = "exceptiongroup", marker = "python_full_version < '3.11'" },
{ name = "jedi", marker = "python_full_version < '3.11'" },
{ name = "matplotlib-inline", marker = "python_full_version < '3.11'" },
{ name = "pexpect", marker = "python_full_version < '3.11' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
{ name = "prompt-toolkit", marker = "python_full_version < '3.11'" },
{ name = "pygments", marker = "python_full_version < '3.11'" },
{ name = "stack-data", marker = "python_full_version < '3.11'" },
{ name = "traitlets", marker = "python_full_version < '3.11'" },
{ name = "typing-extensions", marker = "python_full_version < '3.11'" },
{ name = "colorama", marker = "sys_platform == 'win32'" },
{ name = "decorator" },
{ name = "exceptiongroup" },
{ name = "jedi" },
{ name = "matplotlib-inline" },
{ name = "pexpect", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
{ name = "prompt-toolkit" },
{ name = "pygments" },
{ name = "stack-data" },
{ name = "traitlets" },
{ name = "typing-extensions" },
]
sdist = { url = "https://files.pythonhosted.org/packages/40/18/f8598d287006885e7136451fdea0755af4ebcbfe342836f24deefaed1164/ipython-8.39.0.tar.gz", hash = "sha256:4110ae96012c379b8b6db898a07e186c40a2a1ef5d57a7fa83166047d9da7624", size = 5513971, upload-time = "2026-03-27T10:02:13.94Z" }
wheels = [
@@ -1583,18 +1586,18 @@ resolution-markers = [
"python_full_version == '3.11.*'",
]
dependencies = [
{ name = "colorama", marker = "python_full_version >= '3.11' and sys_platform == 'win32'" },
{ name = "decorator", marker = "python_full_version >= '3.11'" },
{ name = "ipython-pygments-lexers", marker = "python_full_version >= '3.11'" },
{ name = "jedi", marker = "python_full_version >= '3.11'" },
{ name = "matplotlib-inline", marker = "python_full_version >= '3.11'" },
{ name = "pexpect", marker = "python_full_version >= '3.11' and sys_platform != 'emscripten' and sys_platform != 'win32'" },
{ name = "prompt-toolkit", marker = "python_full_version >= '3.11'" },
{ name = "psutil", marker = "python_full_version >= '3.11' and sys_platform != 'cygwin' and sys_platform != 'emscripten'" },
{ name = "pygments", marker = "python_full_version >= '3.11'" },
{ name = "stack-data", marker = "python_full_version >= '3.11'" },
{ name = "traitlets", marker = "python_full_version >= '3.11'" },
{ name = "typing-extensions", marker = "python_full_version == '3.11.*'" },
{ name = "colorama", marker = "sys_platform == 'win32'" },
{ name = "decorator" },
{ name = "ipython-pygments-lexers" },
{ name = "jedi" },
{ name = "matplotlib-inline" },
{ name = "pexpect", marker = "sys_platform != 'emscripten' and sys_platform != 'win32'" },
{ name = "prompt-toolkit" },
{ name = "psutil", marker = "sys_platform != 'cygwin' and sys_platform != 'emscripten'" },
{ name = "pygments" },
{ name = "stack-data" },
{ name = "traitlets" },
{ name = "typing-extensions", marker = "python_full_version < '3.12'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/53/59/165d3b4d75cc34add3122c4417ecb229085140ac573103c223cd01dde96f/ipython-9.15.0.tar.gz", hash = "sha256:da2819ce2aa83135257df830660b1176d986c3d2876db24df01974fa955b2756", size = 4442580, upload-time = "2026-06-26T11:03:35.913Z" }
wheels = [
@@ -1606,7 +1609,7 @@ name = "ipython-pygments-lexers"
version = "1.1.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "pygments", marker = "python_full_version >= '3.11'" },
{ name = "pygments" },
]
sdist = { url = "https://files.pythonhosted.org/packages/ef/4c/5dd1d8af08107f88c7f741ead7a40854b8ac24ddf9ae850afbcf698aa552/ipython_pygments_lexers-1.1.1.tar.gz", hash = "sha256:09c0138009e56b6854f9535736f4171d855c8c08a563a0dcd8022f78355c7e81", size = 8393, upload-time = "2025-01-17T11:24:34.505Z" }
wheels = [
@@ -2858,7 +2861,7 @@ name = "nvidia-cudnn-cu13"
version = "9.19.0.56"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-cublas", marker = "(python_full_version < '3.14' and sys_platform == 'emscripten') or (python_full_version < '3.14' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" },
{ name = "nvidia-cublas" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/f1/84/26025437c1e6b61a707442184fa0c03d083b661adf3a3eecfd6d21677740/nvidia_cudnn_cu13-9.19.0.56-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:6ed29ffaee1176c612daf442e4dd6cfeb6a0caa43ddcbeb59da94953030b1be4", size = 433781201, upload-time = "2026-02-03T20:40:53.805Z" },
@@ -2870,7 +2873,7 @@ name = "nvidia-cufft"
version = "12.0.0.61"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-nvjitlink", marker = "(python_full_version < '3.14' and sys_platform == 'emscripten') or (python_full_version < '3.14' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" },
{ name = "nvidia-nvjitlink" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/8b/ae/f417a75c0259e85c1d2f83ca4e960289a5f814ed0cea74d18c353d3e989d/nvidia_cufft-12.0.0.61-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:2708c852ef8cd89d1d2068bdbece0aa188813a0c934db3779b9b1faa8442e5f5", size = 214053554, upload-time = "2025-09-04T08:31:38.196Z" },
@@ -2900,9 +2903,9 @@ name = "nvidia-cusolver"
version = "12.0.4.66"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-cublas", marker = "(python_full_version < '3.14' and sys_platform == 'emscripten') or (python_full_version < '3.14' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" },
{ name = "nvidia-cusparse", marker = "(python_full_version < '3.14' and sys_platform == 'emscripten') or (python_full_version < '3.14' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" },
{ name = "nvidia-nvjitlink", marker = "(python_full_version < '3.14' and sys_platform == 'emscripten') or (python_full_version < '3.14' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" },
{ name = "nvidia-cublas" },
{ name = "nvidia-cusparse" },
{ name = "nvidia-nvjitlink" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/c8/c3/b30c9e935fc01e3da443ec0116ed1b2a009bb867f5324d3f2d7e533e776b/nvidia_cusolver-12.0.4.66-py3-none-manylinux_2_27_aarch64.whl", hash = "sha256:02c2457eaa9e39de20f880f4bd8820e6a1cfb9f9a34f820eb12a155aa5bc92d2", size = 223467760, upload-time = "2025-09-04T08:33:04.222Z" },
@@ -2914,7 +2917,7 @@ name = "nvidia-cusparse"
version = "12.6.3.3"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "nvidia-nvjitlink", marker = "(python_full_version < '3.14' and sys_platform == 'emscripten') or (python_full_version < '3.14' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" },
{ name = "nvidia-nvjitlink" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/f8/94/5c26f33738ae35276672f12615a64bd008ed5be6d1ebcb23579285d960a9/nvidia_cusparse-12.6.3.3-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:80bcc4662f23f1054ee334a15c72b8940402975e0eab63178fc7e670aa59472c", size = 162155568, upload-time = "2025-09-04T08:33:42.864Z" },
@@ -3091,10 +3094,10 @@ resolution-markers = [
"python_full_version < '3.11'",
]
dependencies = [
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
{ name = "python-dateutil", marker = "python_full_version < '3.11'" },
{ name = "pytz", marker = "python_full_version < '3.11'" },
{ name = "tzdata", marker = "python_full_version < '3.11'" },
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" } },
{ name = "python-dateutil" },
{ name = "pytz" },
{ name = "tzdata" },
]
sdist = { url = "https://files.pythonhosted.org/packages/9c/d6/9f8431bacc2e19dca897724cd097b1bb224a6ad5433784a44b587c7c13af/pandas-2.2.3.tar.gz", hash = "sha256:4f18ba62b61d7e192368b84517265a99b4d7ee8912f8708660fb4a366cc82667", size = 4399213, upload-time = "2024-09-20T13:10:04.827Z" }
wheels = [
@@ -3143,11 +3146,11 @@ resolution-markers = [
"python_full_version == '3.11.*'",
]
dependencies = [
{ name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" },
{ name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12' or python_full_version >= '3.14'" },
{ name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12' and python_full_version < '3.14'" },
{ name = "python-dateutil", marker = "python_full_version >= '3.11' and python_full_version < '3.14'" },
{ name = "pytz", marker = "python_full_version >= '3.11' and python_full_version < '3.14'" },
{ name = "tzdata", marker = "python_full_version >= '3.11' and python_full_version < '3.14'" },
{ name = "python-dateutil" },
{ name = "pytz" },
{ name = "tzdata" },
]
sdist = { url = "https://files.pythonhosted.org/packages/33/01/d40b85317f86cf08d853a4f495195c73815fdf205eef3993821720274518/pandas-2.3.3.tar.gz", hash = "sha256:e05e1af93b977f7eafa636d043f9f94c7ee3ac81af99c13508215942e64c993b", size = 4495223, upload-time = "2025-09-29T23:34:51.853Z" }
wheels = [
@@ -3210,9 +3213,9 @@ resolution-markers = [
"python_full_version >= '3.14' and sys_platform != 'emscripten' and sys_platform != 'win32'",
]
dependencies = [
{ name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.14'" },
{ name = "python-dateutil", marker = "python_full_version >= '3.14'" },
{ name = "tzdata", marker = "(python_full_version >= '3.14' and sys_platform == 'emscripten') or (python_full_version >= '3.14' and sys_platform == 'win32')" },
{ name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" } },
{ name = "python-dateutil" },
{ name = "tzdata", marker = "sys_platform == 'emscripten' or sys_platform == 'win32'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/f8/87/4341c6252d1c47b08768c3d25ac487362bf403f0313ddae4a2a26c9b1b4c/pandas-3.0.3.tar.gz", hash = "sha256:696a4a00a2a2a35d4e5deb3fc946641b96c944f02230e4f76137fe35d806c4fc", size = 4651414, upload-time = "2026-05-11T18:54:29.21Z" }
wheels = [
@@ -3320,7 +3323,7 @@ name = "pexpect"
version = "4.9.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "ptyprocess", marker = "(python_full_version < '3.14' and sys_platform == 'emscripten') or (python_full_version < '3.14' and sys_platform == 'win32') or (sys_platform != 'emscripten' and sys_platform != 'win32')" },
{ name = "ptyprocess" },
]
sdist = { url = "https://files.pythonhosted.org/packages/42/92/cc564bf6381ff43ce1f4d06852fc19a2f11d180f23dc32d9588bee2f149d/pexpect-4.9.0.tar.gz", hash = "sha256:ee7d41123f3c9911050ea2c2dac107568dc43b2d3b0c7557a33212c398ead30f", size = 166450, upload-time = "2023-11-25T09:07:26.339Z" }
wheels = [
@@ -3912,8 +3915,8 @@ crypto = [
[[package]]
name = "pylance"
version = "7.0.0"
source = { registry = "https://pypi.org/simple" }
version = "9.0.0rc1"
source = { registry = "https://pypi.fury.io/lance-format/" }
dependencies = [
{ name = "lance-namespace" },
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
@@ -3922,12 +3925,12 @@ dependencies = [
{ name = "pyarrow" },
]
wheels = [
{ url = "https://files.pythonhosted.org/packages/ac/ad/2f64921bf346e7075aef24a72595db44821724a3d89a9a92dd24e79632aa/pylance-7.0.0-cp39-abi3-macosx_11_0_arm64.whl", hash = "sha256:98422021975be76e72b1572f41b8c9abb3bee5bdc9bfa5e9ce731110a65ed4d1", size = 62134146, upload-time = "2026-05-27T21:59:37.459Z" },
{ url = "https://files.pythonhosted.org/packages/73/1c/c5a01bee0160b55d9a98895cbd33091d038f0a0995b121ab72e629008d02/pylance-7.0.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4bec86ee5b6fbd8bfc493e653f0a1fba0303cfe5492b9b46fc25ab908edc7183", size = 65373684, upload-time = "2026-05-27T22:04:01.584Z" },
{ url = "https://files.pythonhosted.org/packages/eb/da/1fe8b8f7dbfe734d76af76acc994fc360a0d0c79a4874ef69f5a72a58fe3/pylance-7.0.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:881491432c53184e52f8d1db8d5f872f39a03f36fb104bec77b33d379519d8b5", size = 69458555, upload-time = "2026-05-27T22:16:50.567Z" },
{ url = "https://files.pythonhosted.org/packages/76/f0/dd505cf3fd0226ab9d94759acd713125af1d3bfacfd80bbd52e3b9f89509/pylance-7.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:18453999e7fff4f76b16d6b7882c9df0628bd142ff95e2461bd7dd5ee3fe0af3", size = 65394430, upload-time = "2026-05-27T22:05:30.923Z" },
{ url = "https://files.pythonhosted.org/packages/17/ba/2357b81034f28eb00790e258ed140289a6a887a7468ca9df6349fd186b27/pylance-7.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:04a58051d408c60fe76d41a220dcaf8fea8fb6d1aa0ca78a709b60bc3cc8d19a", size = 69473470, upload-time = "2026-05-27T22:17:18.935Z" },
{ url = "https://files.pythonhosted.org/packages/1f/ec/5c00b6303a67d787f9475141832cbdc513d674ac3dcaeef8a7b169905e65/pylance-7.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:467d4864af047eaab4e1370e2f1e88e2c6f507c079874421116cb41d78bc3629", size = 74792863, upload-time = "2026-05-27T22:19:23.875Z" },
{ url = "https://pypi.fury.io/lance-format/-/ver_vEHBE/pylance-9.0.0rc1-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:f0b6b02a1808bb3072ee7fe4e36614cae6f86302513e73ec7f55b2234a963b24" },
{ url = "https://pypi.fury.io/lance-format/-/ver_1Jipm4/pylance-9.0.0rc1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:30f0ebf0d88034301819eb964f9236ce555aaa58e7ab89c5975a3e2250bbb405" },
{ url = "https://pypi.fury.io/lance-format/-/ver_IvKxo/pylance-9.0.0rc1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:44609ea2615ea6e684b85478d1694af2026458f61cf7895ecc75e238bfd17aa8" },
{ url = "https://pypi.fury.io/lance-format/-/ver_2hidj1/pylance-9.0.0rc1-cp310-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:182167a8dba9eeabffbffd53bd5b8548613d4d459b7cd7b34a840dd00cbb806f" },
{ url = "https://pypi.fury.io/lance-format/-/ver_1dFx3r/pylance-9.0.0rc1-cp310-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:8a63b11e814b7eab758bcaf0d6f97eb05ea86203d9fb0af718c462c24c7d6c9c" },
{ url = "https://pypi.fury.io/lance-format/-/ver_2a8dSh/pylance-9.0.0rc1-cp310-abi3-win_amd64.whl", hash = "sha256:2ff8b953ae2b0550490c1a7efd210aa91bc223d200ffac28849056cfd7436d97" },
]
[[package]]
@@ -4683,10 +4686,10 @@ resolution-markers = [
"python_full_version < '3.11'",
]
dependencies = [
{ name = "joblib", marker = "python_full_version < '3.11'" },
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
{ name = "scipy", version = "1.15.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
{ name = "threadpoolctl", marker = "python_full_version < '3.11'" },
{ name = "joblib" },
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" } },
{ name = "scipy", version = "1.15.3", source = { registry = "https://pypi.org/simple" } },
{ name = "threadpoolctl" },
]
sdist = { url = "https://files.pythonhosted.org/packages/98/c2/a7855e41c9d285dfe86dc50b250978105dce513d6e459ea66a6aeb0e1e0c/scikit_learn-1.7.2.tar.gz", hash = "sha256:20e9e49ecd130598f1ca38a1d85090e1a600147b9c02fa6f15d69cb53d968fda", size = 7193136, upload-time = "2025-09-09T08:21:29.075Z" }
wheels = [
@@ -4734,13 +4737,13 @@ resolution-markers = [
"python_full_version == '3.11.*'",
]
dependencies = [
{ name = "joblib", marker = "python_full_version >= '3.11'" },
{ name = "narwhals", marker = "python_full_version >= '3.11'" },
{ name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" },
{ name = "joblib" },
{ name = "narwhals" },
{ name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },
{ name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },
{ name = "scipy", version = "1.17.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" },
{ name = "scipy", version = "1.17.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },
{ name = "scipy", version = "1.18.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },
{ name = "threadpoolctl", marker = "python_full_version >= '3.11'" },
{ name = "threadpoolctl" },
]
sdist = { url = "https://files.pythonhosted.org/packages/fa/6f/37092bdb25f712817231799fc5674d8e704066a8a70c1d2d40517e18b4ab/scikit_learn-1.9.0.tar.gz", hash = "sha256:8833266989d3a5110178a9fae30783675460724d0e1efb13b14901d2c660c557", size = 7750767, upload-time = "2026-06-02T11:54:32.706Z" }
wheels = [
@@ -4784,7 +4787,7 @@ resolution-markers = [
"python_full_version < '3.11'",
]
dependencies = [
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" },
{ name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" } },
]
sdist = { url = "https://files.pythonhosted.org/packages/0f/37/6964b830433e654ec7485e45a00fc9a27cf868d622838f6b6d9c5ec0d532/scipy-1.15.3.tar.gz", hash = "sha256:eae3cf522bc7df64b42cad3925c876e1b0b6c35c1337c93e12c0f366f55b0eaf", size = 59419214, upload-time = "2025-05-08T16:13:05.955Z" }
wheels = [
@@ -4843,7 +4846,7 @@ resolution-markers = [
"python_full_version == '3.11.*'",
]
dependencies = [
{ name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" },
{ name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" } },
]
sdist = { url = "https://files.pythonhosted.org/packages/7a/97/5a3609c4f8d58b039179648e62dd220f89864f56f7357f5d4f45c29eb2cc/scipy-1.17.1.tar.gz", hash = "sha256:95d8e012d8cb8816c226aef832200b1d45109ed4464303e997c5b13122b297c0", size = 30573822, upload-time = "2026-02-23T00:26:24.851Z" }
wheels = [
@@ -4920,7 +4923,7 @@ resolution-markers = [
"python_full_version >= '3.12' and python_full_version < '3.14'",
]
dependencies = [
{ name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },
{ name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" } },
]
sdist = { url = "https://files.pythonhosted.org/packages/a7/25/c2700dfaf6442b4effaa91af24ebce5dc9d31bb4a69706313aae70d72cd0/scipy-1.18.0.tar.gz", hash = "sha256:67b2ad2ad54c72ca6d04975a9b2df8c3638c34ddd5b28738e94fc2b57929d378", size = 30774447, upload-time = "2026-06-19T15:01:43.456Z" }
wheels = [
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "lancedb"
version = "0.39.0-beta.3"
version = "0.39.0-beta.4"
edition.workspace = true
description = "LanceDB: A serverless, low-latency vector database for AI applications"
license.workspace = true
+59 -1
View File
@@ -24,7 +24,7 @@ use crate::data::scannable::Scannable;
use crate::database::listing::ListingDatabase;
use crate::database::{
CloneTableRequest, Database, DatabaseOptions, JobInfo, OpenTableRequest, ReadConsistency,
TableNamesRequest,
SecretInfo, TableNamesRequest,
};
use crate::embeddings::{EmbeddingRegistry, MemoryRegistry};
use crate::error::{Error, Result};
@@ -586,10 +586,15 @@ impl Connection {
/// Registration is remote-only and always asynchronous. Waiting on the
/// returned typed job yields the durable [`crate::function::FunctionVersion`].
/// Local databases return [`Error::NotSupported`].
///
/// The request's binding shape is validated here rather than in any one
/// language binding, so every client surface rejects the same envelopes
/// before one reaches the wire.
pub async fn create_function_async(
&self,
request: crate::function::FunctionRegistrationRequest,
) -> Result<crate::job::Job<crate::function::FunctionVersion>> {
request.validate()?;
self.internal.create_function_async(request).await
}
@@ -645,6 +650,59 @@ impl Connection {
.await
}
/// Create a named Secret in this database.
///
/// Fails if the name is taken, so a create can never silently become a
/// rotation. There is no API that reads a stored credential back; the only
/// consumer is a Function that binds the Secret by name. Local databases
/// return [`Error::NotSupported`].
pub async fn create_secret(&self, name: impl AsRef<str>, value: impl AsRef<str>) -> Result<()> {
let value = value.as_ref();
crate::function::validate_secret_value(value)?;
self.internal.create_secret(name.as_ref(), value).await
}
/// Replace the credential behind an existing Secret.
///
/// Fails if it does not exist. Every Function bound to the Secret resolves
/// the new value from its next execution, and no new Function version is
/// minted -- which is what lets a rotation reach columns pinned to a
/// version registered before it. Local databases return
/// [`Error::NotSupported`].
pub async fn alter_secret(&self, name: impl AsRef<str>, value: impl AsRef<str>) -> Result<()> {
let value = value.as_ref();
crate::function::validate_secret_value(value)?;
self.internal.alter_secret(name.as_ref(), value).await
}
/// The names of every Secret in this database.
///
/// Names only. No path in this API returns a stored credential, by
/// construction rather than by policy. Local databases return
/// [`Error::NotSupported`].
pub async fn list_secrets(&self) -> Result<Vec<String>> {
self.internal.list_secrets().await
}
/// Drop a Secret.
///
/// Functions bound to it fail at their next job, naming the Secret; that
/// is the revocation path. The name becomes free to reuse, and a new
/// Secret under it is picked up by everything still bound to that name.
/// Local databases return [`Error::NotSupported`].
pub async fn drop_secret(&self, name: impl AsRef<str>) -> Result<()> {
self.internal.drop_secret(name.as_ref()).await
}
/// What this database records about one Secret: its name and timestamps.
///
/// Never the value. The type it returns has no field for one, so this is a
/// property of the API rather than of what the caller chooses to read.
/// Local databases return [`Error::NotSupported`].
pub async fn describe_secret(&self, name: impl AsRef<str>) -> Result<SecretInfo> {
self.internal.describe_secret(name.as_ref()).await
}
/// Rename a table in the database.
///
/// This is only supported in LanceDB Cloud.
+48
View File
@@ -249,9 +249,29 @@ fn function_catalog_not_supported<T>() -> Result<T> {
})
}
fn secret_catalog_not_supported<T>() -> Result<T> {
Err(crate::error::Error::NotSupported {
message: "Secret operations are not supported by this database".to_string(),
})
}
/// The `Database` trait defines the interface for database implementations.
///
/// A database is responsible for managing tables and their metadata.
/// What a database records about a Secret. Never its value.
///
/// Returned by [`crate::connection::Connection::describe_secret`]. There is no
/// field for the credential and no method that could produce one.
#[derive(Debug, Clone, PartialEq, Eq, serde::Deserialize)]
pub struct SecretInfo {
/// The Secret's database-scoped name.
pub name: String,
/// When the Secret was created, as an RFC 3339 timestamp.
pub created_at: String,
/// When the Secret's value was last rotated, as an RFC 3339 timestamp.
pub updated_at: String,
}
#[async_trait::async_trait]
pub trait Database:
Send + Sync + std::any::Any + std::fmt::Debug + std::fmt::Display + 'static
@@ -317,6 +337,34 @@ pub trait Database:
async fn drop_function(&self, _name: &str, _version: &str) -> Result<bool> {
function_catalog_not_supported()
}
/// Create a named Secret in this database. Fails if the name is taken, so
/// a create can never silently become a rotation.
async fn create_secret(&self, _name: &str, _value: &str) -> Result<()> {
secret_catalog_not_supported()
}
/// Replace the credential behind an existing Secret. Fails if it does not
/// exist. Every Function bound to it resolves the new value from its next
/// execution, with no new Function version.
async fn alter_secret(&self, _name: &str, _value: &str) -> Result<()> {
secret_catalog_not_supported()
}
/// The names of every Secret in this database.
///
/// Names only. No API path returns a stored credential, by construction
/// rather than by policy.
async fn list_secrets(&self) -> Result<Vec<String>> {
secret_catalog_not_supported()
}
/// Drop a Secret. Functions bound to it fail at their next job, which is
/// the revocation path.
async fn drop_secret(&self, _name: &str) -> Result<()> {
secret_catalog_not_supported()
}
/// What the database records about one Secret: its name and timestamps,
/// never its value.
async fn describe_secret(&self, _name: &str) -> Result<SecretInfo> {
secret_catalog_not_supported()
}
/// Open a job by id, returning a handle with its record already
/// populated. Fails with [`crate::Error::JobNotFound`] when the server has
/// no such job.
+115 -1
View File
@@ -5,7 +5,7 @@
//! backend-neutral terminal result of a computed-column refresh.
//!
//! This module contains client/wire values only. Catalog persistence,
//! environment bake, and execution are owned by Sophon.
//! environment bake, secret resolution, and execution are owned by Sophon.
use std::collections::BTreeMap;
@@ -409,6 +409,8 @@ pub struct FunctionVersion {
runtime: PythonRuntimeSpec,
runtime_digest: String,
environment_digest: String,
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
secret_env_bindings: BTreeMap<String, String>,
created_at: String,
}
@@ -441,6 +443,16 @@ impl FunctionVersion {
&self.environment_digest
}
/// Declared environment variable name to the Secret each one resolves.
///
/// Bindings are part of this version's identity; the credentials behind
/// them are not, and resolve at execution. Rotating a bound Secret
/// therefore changes what the same version runs with, and no value has a
/// field in this model.
pub fn secret_env_bindings(&self) -> &BTreeMap<String, String> {
&self.secret_env_bindings
}
pub fn created_at(&self) -> &str {
&self.created_at
}
@@ -481,13 +493,115 @@ pub struct FunctionArtifactRequest {
pub adapter: PythonAdapterSpec,
}
/// A Function binds at most this many Secrets to environment variables.
///
/// Each bound Secret is one extra read on the launch path of every fragment, so
/// the count needs a bound for the same reason a credential needs a size limit.
pub const MAX_FUNCTION_SECRET_ENV_BINDINGS: usize = 16;
/// Largest credential a Secret may hold, matching the limit the service
/// enforces. Bounded because the value is destined for a process environment.
pub const MAX_SECRET_VALUE_BYTES: usize = 64 * 1024;
/// Whether `name` is a portable POSIX environment variable name.
///
/// Leading letter or underscore, then letters, digits, or underscores. Names
/// reserved by the execution sandbox are deliberately not checked here: that
/// list belongs to the runtime that owns it, and a copy in the client would
/// drift from it silently.
fn is_portable_env_name(name: &str) -> bool {
let mut bytes = name.bytes();
bytes
.next()
.is_some_and(|byte| byte == b'_' || byte.is_ascii_alphabetic())
&& bytes.all(|byte| byte == b'_' || byte.is_ascii_alphanumeric())
}
/// Reject a credential the service would refuse on size alone.
///
/// Checked before the request body is built, so an oversized value is never
/// serialized or uploaded.
pub(crate) fn validate_secret_value(value: &str) -> Result<()> {
if value.is_empty() {
return Err(Error::InvalidInput {
message: "a Secret value must not be empty".to_string(),
});
}
if value.contains('\0') {
return Err(Error::InvalidInput {
message: "a Secret value must not contain NUL".to_string(),
});
}
if value.len() > MAX_SECRET_VALUE_BYTES {
return Err(Error::InvalidInput {
message: format!(
"a Secret value is at most {MAX_SECRET_VALUE_BYTES} bytes, not {}",
value.len()
),
});
}
Ok(())
}
/// Stable request envelope for remote immutable Function registration.
///
/// Credential values deliberately have no field here. The only secret-shaped
/// thing a client sends is `secret_env_bindings`: the name of a Secret the
/// database already holds, which Sophon resolves inside the remote runtime.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct FunctionRegistrationRequest {
pub name: String,
pub artifact: FunctionArtifactRequest,
pub signature: FunctionSignature,
pub runtime: PythonRuntimeSpec,
/// Declared environment variable name to the Secret it binds. A binding is
/// a reference: whether the Secret exists is answered when a column is
/// declared against this version, not here.
#[serde(default, skip_serializing_if = "BTreeMap::is_empty")]
pub secret_env_bindings: BTreeMap<String, String>,
}
impl FunctionRegistrationRequest {
/// Reject a registration whose bindings exceed what a launch can deliver.
///
/// Shape only, and deliberately not a check that each bound Secret exists:
/// that is the service's answer, and it is asked for the first time when a
/// column is declared against the registered version.
pub fn validate(&self) -> Result<()> {
if self.secret_env_bindings.len() > MAX_FUNCTION_SECRET_ENV_BINDINGS {
return Err(Error::InvalidInput {
message: format!(
"a Function binds at most {MAX_FUNCTION_SECRET_ENV_BINDINGS} secrets, not {}",
self.secret_env_bindings.len()
),
});
}
for variable in self.secret_env_bindings.keys() {
if !is_portable_env_name(variable) {
return Err(Error::InvalidInput {
message: format!(
"secret_env_bindings key '{variable}' is not a portable \
environment variable name"
),
});
}
// `env` travels with the Function and is readable wherever its
// record is; a bound Secret is not. One name carrying both would
// resolve by delivery order, so refuse rather than pick.
if self
.runtime
.env()
.is_some_and(|env| env.contains_key(variable))
{
return Err(Error::InvalidInput {
message: format!(
"secret_env_bindings key '{variable}' is already set by runtime.env"
),
});
}
}
Ok(())
}
}
impl_json!(FunctionRegistrationRequest);
+76 -7
View File
@@ -404,6 +404,18 @@ fn validate_dns_hostname(hostname: &str) -> Result<()> {
Ok(())
}
/// Whether a route's request body is a credential rather than a description of
/// one.
///
/// Matched on the path segment rather than a versioned prefix, so a `/v2/` bump
/// or a route added under the namespace later is covered without anyone
/// remembering to extend this. Every secrets route is denied, not only the two
/// that carry a value: their bodies hold names and page tokens, which are worth
/// nothing in a debug log next to the risk of a new verb landing here unnoticed.
fn route_carries_credential(path: &str) -> bool {
path.split('/').any(|segment| segment == "secrets")
}
impl RestfulLanceDbClient<Sender> {
fn get_timeout(passed: Option<Duration>, env_var: &str) -> Result<Option<Duration>> {
if let Some(passed) = passed {
@@ -610,12 +622,14 @@ impl<S: HttpSend> RestfulLanceDbClient<S> {
) -> Result<HeaderMap> {
let mut headers = HeaderMap::new();
if !api_key.is_empty() {
headers.insert(
HeaderName::from_static("x-api-key"),
HeaderValue::from_str(api_key).map_err(|_| Error::InvalidInput {
message: "non-ascii api key provided".to_string(),
})?,
);
// `log_request` prints the request's Debug, which prints headers.
// Marking the value sensitive is what makes that print `Sensitive`
// instead of the key itself.
let mut key = HeaderValue::from_str(api_key).map_err(|_| Error::InvalidInput {
message: "non-ascii api key provided".to_string(),
})?;
key.set_sensitive(true);
headers.insert(HeaderName::from_static("x-api-key"), key);
}
if region == "local" {
let host = format!("{}.local.api.lancedb.com", db_name);
@@ -845,7 +859,12 @@ impl<S: HttpSend> RestfulLanceDbClient<S> {
.headers()
.get("content-type")
.map(|v| v.to_str().unwrap());
if content_type == Some("application/json") {
if route_carries_credential(request.url().path()) {
debug!(
"Sending request_id={}: {:?} with body suppressed",
request_id, request
);
} else if content_type == Some("application/json") {
let body = request.body().as_ref().unwrap().as_bytes().unwrap();
let body = String::from_utf8_lossy(body);
debug!(
@@ -1192,6 +1211,56 @@ mod tests {
assert_eq!(headers.get("x-api-key").unwrap(), "api-key");
}
/// `log_request` prints the request's Debug, and Debug for a request prints
/// its headers. Marking the value sensitive is the only thing standing
/// between the API key and every debug line; assert on the header map's own
/// Debug, which is what that printing reduces to.
#[test]
fn test_api_key_is_redacted_in_debug_output() {
let headers = RestfulLanceDbClient::<Sender>::default_headers(
"sk-live-sentinel",
"us-east-1",
"db-name",
false,
&RemoteOptions::default(),
None,
&ClientConfig::default(),
)
.unwrap();
assert_eq!(headers.get("x-api-key").unwrap(), "sk-live-sentinel");
assert!(
!format!("{:?}", headers).contains("sk-live-sentinel"),
"the API key must not survive Debug formatting"
);
}
/// Denial follows the path segment, so a verb that does not exist yet and a
/// future API version are both covered without an edit here.
#[test]
fn test_secrets_routes_never_log_a_body() {
for route in [
"/v1/secrets/create",
"/v1/secrets/alter",
"/v1/secrets/list",
"/v1/secrets/drop",
"/v1/secrets/describe",
"/v2/secrets/rotate",
] {
assert!(
route_carries_credential(route),
"{route} must never log a body"
);
}
for route in [
"/v1/functions/create",
"/v1/table/foo/query",
"/v1/jobs/list",
] {
assert!(!route_carries_credential(route), "{route} is not a secret");
}
}
#[test]
fn test_rejects_invalid_cloud_dns_hostname() {
let invalid_database_names = ["a".repeat(64), "invalid..database".to_string()];
+166 -1
View File
@@ -21,7 +21,7 @@ use lance_namespace::models::{
use crate::Error;
use crate::database::{
CloneTableRequest, CreateTableMode, CreateTableRequest, Database, DatabaseOptions, JobInfo,
OpenTableRequest, ReadConsistency, TableNamesRequest,
OpenTableRequest, ReadConsistency, SecretInfo, TableNamesRequest,
};
use crate::error::Result;
use crate::function::{FunctionRegistrationRequest, FunctionVersion};
@@ -352,6 +352,20 @@ impl RemoteDatabase {
}
impl<S: HttpSend> RemoteDatabase<S> {
/// `create` and `alter` differ only in which name state the server
/// requires, so they share one request shape. The value is a request field
/// and never a path segment or query parameter, which keeps it out of
/// access logs and proxy traces.
async fn write_secret(&self, route: &str, name: &str, value: &str) -> Result<()> {
let req = self.client.post(route).json(&serde_json::json!({
"name": name,
"value": value,
}));
let (request_id, response) = self.client.send(req).await?;
self.client.check_response(&request_id, response).await?;
Ok(())
}
async fn submit_drop_table(
&self,
name: &str,
@@ -570,6 +584,21 @@ struct RemoteDropFunctionResponse {
dropped: bool,
}
#[derive(serde::Deserialize)]
struct RemoteListSecretsResponse {
#[serde(default)]
secrets: Vec<RemoteListedSecret>,
#[serde(default)]
page_token: Option<String>,
}
/// An object rather than a bare name so a later listing can carry a Secret's
/// type or last-updated time without breaking this one.
#[derive(serde::Deserialize)]
struct RemoteListedSecret {
name: String,
}
/// Bound on `list_jobs` page walking; a warning is logged when the listing
/// is truncated at this many pages.
const MAX_LIST_JOBS_PAGES: usize = 100;
@@ -671,6 +700,66 @@ impl<S: HttpSend> Database for RemoteDatabase<S> {
Ok(response.dropped)
}
async fn create_secret(&self, name: &str, value: &str) -> Result<()> {
self.write_secret("/v1/secrets/create", name, value).await
}
async fn alter_secret(&self, name: &str, value: &str) -> Result<()> {
self.write_secret("/v1/secrets/alter", name, value).await
}
async fn list_secrets(&self) -> Result<Vec<String>> {
let mut names = Vec::new();
let mut page_token: Option<String> = None;
let mut seen_page_tokens = HashSet::new();
loop {
let mut body = serde_json::json!({});
if let Some(token) = &page_token {
body["page_token"] = serde_json::Value::String(token.clone());
}
let req = self.client.post("/v1/secrets/list").json(&body);
let (request_id, response) = self.client.send(req).await?;
let response = self.client.check_response(&request_id, response).await?;
let status = response.status();
let response: RemoteListSecretsResponse =
response.json().await.err_to_http(request_id.clone())?;
names.extend(response.secrets.into_iter().map(|secret| secret.name));
let Some(next_page_token) = response.page_token.filter(|token| !token.is_empty())
else {
break;
};
if !seen_page_tokens.insert(next_page_token.clone()) {
return Err(Error::Http {
source: "Secret listing response repeated a page_token".into(),
request_id,
status_code: Some(status),
});
}
page_token = Some(next_page_token);
}
Ok(names)
}
async fn drop_secret(&self, name: &str) -> Result<()> {
let req = self
.client
.post("/v1/secrets/drop")
.json(&serde_json::json!({ "name": name }));
let (request_id, response) = self.client.send(req).await?;
self.client.check_response(&request_id, response).await?;
Ok(())
}
async fn describe_secret(&self, name: &str) -> Result<SecretInfo> {
let req = self
.client
.post("/v1/secrets/describe")
.json(&serde_json::json!({ "name": name }));
let (request_id, response) = self.client.send(req).await?;
let response = self.client.check_response(&request_id, response).await?;
response.json().await.err_to_http(request_id)
}
async fn open_job(&self, job_id: &str) -> Result<Job> {
let handle = super::job::RemoteJob::new(self.client.clone(), job_id.to_string());
match crate::job::JobHandle::describe(&handle).await {
@@ -2781,6 +2870,82 @@ mod tests {
assert_eq!(batches[0].schema(), schema);
}
#[tokio::test]
async fn test_create_and_alter_secret_send_the_value_in_the_request_body() {
for (route, call) in [("/v1/secrets/create", true), ("/v1/secrets/alter", false)] {
let conn = Connection::new_with_handler(move |request| {
assert_eq!(request.method(), &reqwest::Method::POST);
assert_eq!(request.url().path(), route);
// Never a path segment or query parameter, which is what keeps
// it out of access logs and proxy traces.
assert!(request.url().query().is_none(), "{:?}", request.url());
let body: serde_json::Value =
serde_json::from_slice(request.body().unwrap().as_bytes().unwrap()).unwrap();
assert_eq!(body["name"], "openai-prod");
assert_eq!(body["value"], "sk-live-0001");
http::Response::builder().status(200).body("{}").unwrap()
});
if call {
conn.create_secret("openai-prod", "sk-live-0001")
.await
.unwrap();
} else {
conn.alter_secret("openai-prod", "sk-live-0001")
.await
.unwrap();
}
}
}
#[tokio::test]
async fn test_list_secrets_walks_pages_and_returns_names_only() {
let conn = Connection::new_with_handler(|request| {
assert_eq!(request.url().path(), "/v1/secrets/list");
let body: serde_json::Value =
serde_json::from_slice(request.body().unwrap().as_bytes().unwrap()).unwrap();
let page = body.get("page_token").and_then(|token| token.as_str());
let body = match page {
None => r#"{"secrets":[{"name":"openai-prod"}],"page_token":"p2"}"#,
Some("p2") => r#"{"secrets":[{"name":"hf-prod"}]}"#,
Some(other) => panic!("unexpected page token: {other}"),
};
http::Response::builder().status(200).body(body).unwrap()
});
assert_eq!(
conn.list_secrets().await.unwrap(),
vec!["openai-prod".to_string(), "hf-prod".to_string()]
);
}
/// A server that keeps handing back the same token would otherwise spin
/// forever.
#[tokio::test]
async fn test_list_secrets_rejects_a_repeated_page_token() {
let conn = Connection::new_with_handler(|_| {
http::Response::builder()
.status(200)
.body(r#"{"secrets":[{"name":"openai-prod"}],"page_token":"same"}"#)
.unwrap()
});
let error = conn.list_secrets().await.unwrap_err();
assert!(
error.to_string().contains("repeated a page_token"),
"{error}"
);
}
#[tokio::test]
async fn test_drop_secret_posts_the_name_alone() {
let conn = Connection::new_with_handler(|request| {
assert_eq!(request.url().path(), "/v1/secrets/drop");
let body: serde_json::Value =
serde_json::from_slice(request.body().unwrap().as_bytes().unwrap()).unwrap();
assert_eq!(body, serde_json::json!({"name": "openai-prod"}));
http::Response::builder().status(200).body("{}").unwrap()
});
conn.drop_secret("openai-prod").await.unwrap();
}
#[tokio::test]
async fn test_create_function_async_sends_canonical_request_and_decodes_typed_job() {
const REQUEST: &str = include_str!(
+1 -1
View File
@@ -8021,7 +8021,7 @@ mod tests {
match request.url().path() {
"/v1/table/my_table/backfill_column" => http::Response::builder()
.status(202)
.body(r#"{"job_id": "j-42"}"#.as_bytes().to_vec())
.body(br#"{"job_id": "j-42"}"#.to_vec())
.unwrap(),
"/v1/jobs/describe" => http::Response::builder()
.status(200)
@@ -5,12 +5,17 @@
//!
//! [`super::cast::cast_to_table_schema`] calls [`coerce_blob_expr`].
use std::fmt;
use std::hash::{Hash, Hasher};
use std::sync::Arc;
use arrow_schema::{DataType, Field, FieldRef, Fields};
use arrow_array::{Array, BooleanArray, RecordBatch};
use arrow_schema::{DataType, Field, FieldRef, Fields, Schema};
use arrow_select::nullif::nullif;
use datafusion::functions::core::{get_field, named_struct};
use datafusion_common::ScalarValue;
use datafusion_common::config::ConfigOptions;
use datafusion_expr::ColumnarValue;
use datafusion_physical_expr::ScalarFunctionExpr;
use datafusion_physical_expr::expressions::{CastExpr, Literal};
use datafusion_physical_plan::PhysicalExpr;
@@ -133,16 +138,102 @@ pub(super) fn coerce_blob_expr(
ns_args.push(value);
}
let expr: Arc<dyn PhysicalExpr> = Arc::new(ScalarFunctionExpr::new(
let built: Arc<dyn PhysicalExpr> = Arc::new(ScalarFunctionExpr::new(
&format!("named_struct({})", table_field.name()),
named_struct(),
ns_args,
table_field.clone(),
config.clone(),
));
// `named_struct` always yields a valid struct, so a null input would land
// as a row that set neither `data` nor `uri` -- not an absent blob but a
// malformed one, which Lance rejects on write.
let expr: Arc<dyn PhysicalExpr> = Arc::new(AbsentBlobIsNull {
source: input_expr,
built,
field: table_field.clone(),
});
Ok((expr, table_field.clone()))
}
/// Carries the source column's nullity onto the struct built for it.
///
/// This is its own expression rather than a `CASE` because the projection
/// takes its output field from `return_field`, and the generic implementation
/// rebuilds a bare field -- which would drop the `lance.blob.v2` extension
/// metadata and stop the column being recognised as a blob at all.
#[derive(Debug, Clone)]
struct AbsentBlobIsNull {
source: Arc<dyn PhysicalExpr>,
built: Arc<dyn PhysicalExpr>,
field: FieldRef,
}
impl fmt::Display for AbsentBlobIsNull {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "absent_blob_is_null({}, {})", self.source, self.built)
}
}
impl PartialEq for AbsentBlobIsNull {
fn eq(&self, other: &Self) -> bool {
self.source.eq(&other.source) && self.built.eq(&other.built) && self.field == other.field
}
}
impl Eq for AbsentBlobIsNull {}
impl Hash for AbsentBlobIsNull {
fn hash<H: Hasher>(&self, state: &mut H) {
self.source.hash(state);
self.built.hash(state);
self.field.hash(state);
}
}
impl PhysicalExpr for AbsentBlobIsNull {
fn return_field(&self, _input_schema: &Schema) -> datafusion_common::Result<FieldRef> {
Ok(self.field.clone())
}
fn nullable(&self, _input_schema: &Schema) -> datafusion_common::Result<bool> {
Ok(true)
}
fn evaluate(&self, batch: &RecordBatch) -> datafusion_common::Result<ColumnarValue> {
let rows = batch.num_rows();
let built = self.built.evaluate(batch)?.into_array(rows)?;
let source = self.source.evaluate(batch)?.into_array(rows)?;
let Some(nulls) = source.logical_nulls() else {
return Ok(ColumnarValue::Array(built));
};
// `nullif` nulls the rows the mask marks true, which is where the
// source had no value.
let absent = BooleanArray::new(!nulls.inner(), None);
Ok(ColumnarValue::Array(nullif(built.as_ref(), &absent)?))
}
fn children(&self) -> Vec<&Arc<dyn PhysicalExpr>> {
vec![&self.source, &self.built]
}
fn with_new_children(
self: Arc<Self>,
children: Vec<Arc<dyn PhysicalExpr>>,
) -> datafusion_common::Result<Arc<dyn PhysicalExpr>> {
Ok(Arc::new(Self {
source: children[0].clone(),
built: children[1].clone(),
field: self.field.clone(),
}))
}
fn fmt_sql(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{self}")
}
}
enum BlobInputShape<'a> {
Bytes,
String,
@@ -313,6 +404,11 @@ mod tests {
let data = image.column_by_name("data").unwrap();
assert!(!data.is_null(0));
assert!(data.is_null(1));
// The row itself has to be null, not merely a struct whose children
// are. A present-but-empty struct set neither `data` nor `uri`, which
// Lance rejects as malformed rather than reading as an absent blob.
assert!(!image.is_null(0));
assert!(image.is_null(1));
}
#[tokio::test]
@@ -1,6 +1,7 @@
// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: Copyright The LanceDB Authors
use std::collections::BTreeMap;
use std::fs;
use std::path::PathBuf;
@@ -20,6 +21,26 @@ fn job_result(name: &str) -> Value {
serde_json::from_str::<Value>(&fixture(name)).expect("remote Job fixture")["result"].clone()
}
/// No client value models a resolved credential, at any nesting depth.
fn assert_no_secret_values(value: &Value) {
match value {
Value::Object(values) => {
for (key, value) in values {
assert!(
!matches!(
key.as_str(),
"secret_value" | "secret_values" | "resolved_secret" | "resolved_secrets"
),
"client canonical value must not model resolved secret material"
);
assert_no_secret_values(value);
}
}
Value::Array(values) => values.iter().for_each(assert_no_secret_values),
_ => {}
}
}
#[test]
fn function_version_job_result_matches_shared_canonical_golden() {
let result = job_result("remote_function_job.json");
@@ -28,6 +49,10 @@ fn function_version_job_result_matches_shared_canonical_golden() {
assert_eq!(version.name(), "embed");
assert_eq!(version.version(), "fv_01K3EXACT");
assert_eq!(version.runtime_digest(), "sha256:runtime");
assert_eq!(
version.secret_env_bindings(),
&BTreeMap::from([("HF_TOKEN".to_string(), "hf-prod".to_string())])
);
assert_eq!(
version.to_canonical_json().expect("canonical JSON"),
fixture("remote_function_version.canonical.json").trim()
@@ -142,3 +167,40 @@ fn floating_point_application_literals_are_rejected_consistently() {
.contains("floating-point Function literals")
);
}
#[test]
fn canonical_client_values_carry_bindings_and_no_credentials() {
let result = job_result("remote_function_job.json");
let version = FunctionVersion::from_json(&result.to_string()).expect("FunctionVersion result");
let canonical: Value = serde_json::from_str(
&version
.to_canonical_json()
.expect("canonical FunctionVersion"),
)
.expect("canonical JSON");
assert_eq!(
canonical["secret_env_bindings"],
serde_json::json!({"HF_TOKEN": "hf-prod"})
);
assert_no_secret_values(&canonical);
}
/// Every Function registered before Secrets existed serializes unchanged.
#[test]
fn a_version_without_bindings_keeps_the_original_wire_shape() {
let mut result = job_result("remote_function_job.json");
result
.as_object_mut()
.expect("Function version object")
.remove("secret_env_bindings");
let version = FunctionVersion::from_json(&result.to_string()).expect("FunctionVersion result");
assert!(version.secret_env_bindings().is_empty());
assert!(
!version
.to_canonical_json()
.expect("canonical FunctionVersion")
.contains("secret_env_bindings")
);
}
@@ -5,7 +5,10 @@ use std::fs;
use std::path::PathBuf;
use lancedb::Error;
use lancedb::function::FunctionRegistrationRequest;
use lancedb::function::{
FunctionRegistrationRequest, MAX_FUNCTION_SECRET_ENV_BINDINGS, MAX_SECRET_VALUE_BYTES,
};
use serde_json::Value;
fn fixture(name: &str) -> String {
let path = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
@@ -14,6 +17,26 @@ fn fixture(name: &str) -> String {
fs::read_to_string(path).expect("fixture must be readable")
}
/// A registration request never models a resolved credential, at any depth.
fn assert_no_secret_values(value: &Value) {
match value {
Value::Object(values) => {
for (key, value) in values {
assert!(
!matches!(
key.as_str(),
"secret_value" | "secret_values" | "resolved_secret" | "resolved_secrets"
),
"registration requests must not model resolved secret material"
);
assert_no_secret_values(value);
}
}
Value::Array(values) => values.iter().for_each(assert_no_secret_values),
_ => {}
}
}
#[test]
fn registration_request_matches_shared_canonical_golden() {
let request = FunctionRegistrationRequest::from_json(&fixture(
@@ -22,10 +45,45 @@ fn registration_request_matches_shared_canonical_golden() {
.expect("registration request");
assert_eq!(request.name, "normalize_score");
assert_eq!(request.artifact.adapter.kind, "scalar_to_arrow_batch");
// The unchanged path: a Function that binds nothing serializes today's
// bytes, with no `secret_env_bindings` key at all.
assert!(request.secret_env_bindings.is_empty());
assert_eq!(
request.to_canonical_json().expect("canonical request"),
fixture("remote_function_registration_request.canonical.json").trim()
);
let value: Value =
serde_json::from_str(&request.to_canonical_json().expect("canonical request"))
.expect("request JSON");
assert_no_secret_values(&value);
}
/// The same shared golden as the Python suite builds from `@udf(secrets=...)`
/// plus `bind_secrets`, so both clients agree byte for byte on a bound request.
#[test]
fn secret_bound_registration_request_matches_shared_canonical_golden() {
let request = FunctionRegistrationRequest::from_json(&fixture(
"remote_function_secret_registration_request.json",
))
.expect("registration request");
assert_eq!(request.name, "analyze_caption");
assert_eq!(
request.secret_env_bindings,
std::collections::BTreeMap::from([(
"OPENAI_API_KEY".to_string(),
"openai-prod".to_string()
)])
);
assert_eq!(
request.to_canonical_json().expect("canonical request"),
fixture("remote_function_secret_registration_request.canonical.json").trim()
);
let value: Value =
serde_json::from_str(&request.to_canonical_json().expect("canonical request"))
.expect("request JSON");
assert_no_secret_values(&value);
}
#[tokio::test]
@@ -57,3 +115,105 @@ async fn local_function_catalog_operations_return_stable_not_supported() {
));
}
}
/// The cap is enforced above the backend, so every database and every language
/// surface rejects the same envelope. A local connection would otherwise answer
/// `NotSupported` first, which is what makes it the honest probe here.
#[tokio::test]
async fn a_function_binds_at_most_sixteen_secrets() {
let directory = tempfile::tempdir().unwrap();
let connection = lancedb::connect(directory.path().to_str().unwrap())
.execute()
.await
.unwrap();
let mut request = FunctionRegistrationRequest::from_json(&fixture(
"remote_function_registration_request.json",
))
.unwrap();
request.secret_env_bindings = (0..=MAX_FUNCTION_SECRET_ENV_BINDINGS)
.map(|index| (format!("TOKEN_{index}"), format!("secret-{index}")))
.collect();
let error = connection.create_function_async(request).await.unwrap_err();
assert!(matches!(
error,
Error::InvalidInput { message } if message.contains("at most 16 secrets")
));
}
/// The binding contract is enforced above the backend in full, not just its
/// count: a caller that skips a language binding still cannot register a name
/// the runtime could not deliver.
#[tokio::test]
async fn binding_names_are_validated_before_dispatch() {
let directory = tempfile::tempdir().unwrap();
let connection = lancedb::connect(directory.path().to_str().unwrap())
.execute()
.await
.unwrap();
let mut invalid_name = FunctionRegistrationRequest::from_json(&fixture(
"remote_function_registration_request.json",
))
.unwrap();
invalid_name.secret_env_bindings = [("BAD=NAME".to_string(), "openai-prod".to_string())].into();
let error = connection
.create_function_async(invalid_name)
.await
.unwrap_err();
assert!(matches!(
error,
Error::InvalidInput { message } if message.contains("portable")
));
// `env` is readable wherever the Function's record is; a bound Secret is
// not. The same name cannot mean both.
let mut overlapping = FunctionRegistrationRequest::from_json(&fixture(
"remote_function_registration_request.json",
))
.unwrap();
let bound = overlapping
.runtime
.env()
.and_then(|env| env.keys().next().cloned())
.expect("fixture runtime declares env");
overlapping.secret_env_bindings = [(bound.clone(), "openai-prod".to_string())].into();
let error = connection
.create_function_async(overlapping)
.await
.unwrap_err();
assert!(matches!(
error,
Error::InvalidInput { message } if message.contains("already set by runtime.env")
));
}
/// An oversized credential is refused before a body is built, so it is never
/// serialized or uploaded to be refused by the service instead.
#[tokio::test]
async fn an_oversized_secret_value_is_refused_before_the_wire() {
let directory = tempfile::tempdir().unwrap();
let connection = lancedb::connect(directory.path().to_str().unwrap())
.execute()
.await
.unwrap();
for value in ["", &"x".repeat(MAX_SECRET_VALUE_BYTES + 1)] {
let error = connection
.create_secret("openai-prod", value)
.await
.unwrap_err();
assert!(
matches!(error, Error::InvalidInput { .. }),
"expected InvalidInput, got {error:?}"
);
}
// A local database refuses the verb outright, which is what proves the
// size check ran ahead of the backend rather than instead of it.
let error = connection
.create_secret("openai-prod", "x".repeat(MAX_SECRET_VALUE_BYTES))
.await
.unwrap_err();
assert!(matches!(error, Error::NotSupported { .. }));
}
@@ -24,6 +24,7 @@
},
"runtime_digest": "sha256:runtime",
"environment_digest": "sha256:environment",
"secret_env_bindings": {"HF_TOKEN": "hf-prod"},
"created_at": "2026-08-21T00:00:00Z"
},
"future_job": {"trace_id": "trace-1"}
@@ -0,0 +1 @@
{"artifact":{"adapter":{"kind":"scalar_to_arrow_batch","version":1},"content":{"data":"ZnJvbSBfX2Z1dHVyZV9fIGltcG9ydCBhbm5vdGF0aW9ucwoKZGVmIGFuYWx5emVfY2FwdGlvbihjYXB0aW9uOiBzdHIpIC0+IHN0cjoKICAgIHJldHVybiBjYXB0aW9uLnN0cmlwKCkK","encoding":"base64"},"digest":"sha256:800462c9ad15151a80f83f85b8912ff149300c1563e07f58448f099afcd0d077","entrypoint":"analyze_caption","kind":"python_callable"},"name":"analyze_caption","runtime":{"env":{"MODE":"test"},"environment":{"kind":"pip","packages":["openai==3.7.0"]},"kind":"python","python_version":"3.12"},"secret_env_bindings":{"OPENAI_API_KEY":"openai-prod"},"signature":{"inputs":[{"arrow_type":"utf8","name":"caption","nullable":false}],"output":{"arrow_type":"utf8","kind":"scalar","nullable":false}}}
@@ -0,0 +1,46 @@
{
"artifact": {
"adapter": {
"kind": "scalar_to_arrow_batch",
"version": 1
},
"content": {
"data": "ZnJvbSBfX2Z1dHVyZV9fIGltcG9ydCBhbm5vdGF0aW9ucwoKZGVmIGFuYWx5emVfY2FwdGlvbihjYXB0aW9uOiBzdHIpIC0+IHN0cjoKICAgIHJldHVybiBjYXB0aW9uLnN0cmlwKCkK",
"encoding": "base64"
},
"digest": "sha256:800462c9ad15151a80f83f85b8912ff149300c1563e07f58448f099afcd0d077",
"entrypoint": "analyze_caption",
"kind": "python_callable"
},
"name": "analyze_caption",
"runtime": {
"env": {
"MODE": "test"
},
"environment": {
"kind": "pip",
"packages": [
"openai==3.7.0"
]
},
"kind": "python",
"python_version": "3.12"
},
"secret_env_bindings": {
"OPENAI_API_KEY": "openai-prod"
},
"signature": {
"inputs": [
{
"arrow_type": "utf8",
"name": "caption",
"nullable": false
}
],
"output": {
"arrow_type": "utf8",
"kind": "scalar",
"nullable": false
}
}
}
@@ -1 +1 @@
{"artifact":{"digest":"sha256:code","entrypoint":"embed","kind":"python_callable"},"created_at":"2026-08-21T00:00:00Z","environment_digest":"sha256:environment","name":"embed","runtime":{"env":{"TOKENIZERS_PARALLELISM":"false"},"environment":{"kind":"pip","packages":["sentence-transformers>=3"]},"kind":"python","python_version":"3.12"},"runtime_digest":"sha256:runtime","signature":{"inputs":[{"arrow_type":"utf8","name":"text","nullable":true}],"output":{"arrow_type":"list<float32>","kind":"scalar","nullable":false}},"version":"fv_01K3EXACT"}
{"artifact":{"digest":"sha256:code","entrypoint":"embed","kind":"python_callable"},"created_at":"2026-08-21T00:00:00Z","environment_digest":"sha256:environment","name":"embed","runtime":{"env":{"TOKENIZERS_PARALLELISM":"false"},"environment":{"kind":"pip","packages":["sentence-transformers>=3"]},"kind":"python","python_version":"3.12"},"runtime_digest":"sha256:runtime","secret_env_bindings":{"HF_TOKEN":"hf-prod"},"signature":{"inputs":[{"arrow_type":"utf8","name":"text","nullable":true}],"output":{"arrow_type":"list<float32>","kind":"scalar","nullable":false}},"version":"fv_01K3EXACT"}