Commit Graph
8 Commits
Author SHA1 Message Date
Brendan Clement 07e8a8db17 docs: fix copyable Python examples that fail as published
- Full-text search: the multiline query chain was not parenthesized, so
  pasting it raised an IndentationError.
- Pandas async: the example chained .select onto the coroutine an async
  search returns; it now awaits the search first, and its test asserts
  the result.
- Pydantic: the example searched with a string, which needs a full-text
  index or an embedding function the table does not have; it searches
  with a vector, as the page describes, and asserts the model returned.
- PyTorch: the column-selection example selected columns the page's
  table lacks and printed a schema from a batch that is a dict. It now
  builds its own table and prints the batch's columns.
- Data evolution: the setup used the test's tmp_db fixture, an optional
  data argument and an unimported numpy. It now needs only `db`, which
  the page says how to open.
2026-10-09 13:36:49 -07:00
Brendan Clement 0847d8a437 docs: describe how computed columns and materialized views each refresh
The feature engineering overview said that every mechanism separates
declaring from filling and that every refresh leaves current rows alone.
That holds for computed columns only: creating a materialized view
computes its rows, and a local refresh replaces the whole view even when
nothing changed. The overview now compares the two, keeps the
incremental explanation for computed columns, explains the view
lifecycle with its deployment qualification, and says that a deployment
does not refresh a computed column by itself. Its chooser lists Geneva
for Python work until Functions launch.
2026-10-09 13:12:57 -07:00
Brendan Clement e20664740f docs: publish the Function pages only once Functions launch
The two Function pages declare `launch: functions`, and every other page
marks what it says about Functions, so the assembled site leaves them out
until the launch. Before it, the pages that offered Functions for Python
work point Enterprise readers at Geneva instead.

The SQL Function example now uses Arrow type names, which the server
requires, and states the 3.12 default for `python_version`.
2026-10-09 13:07:33 -07:00
Brendan Clement eedde82253 docs: leave a deployment's materialized view refresh mode to its version
Sophon main now refreshes an eligible materialized view incrementally,
appending what an append-only source added since the last refresh, a day
after the change that made every refresh a full rebuild. So the
materialized view page and the REFRESH reference no longer say recent
versions rebuild in full: they say recent versions run views as SQL jobs,
need no stable row ids and take no refresh options, and that a full or
incremental refresh depends on the version and the view's query. COPY is
described with the same "recent versions" wording.
2026-10-08 21:42:20 -07:00
Brendan Clement 6398ff3629 docs: declare where management, jobs, feature engineering and curation apply
Databases, namespaces, jobs and data evolution work in both offerings.
Materialized views, computed columns and column refresh work in the
embedded library and vary on a deployment, which has to enable them.
Views and functions exist only on a deployment that enables them, and
aggregation, joins, windowing and sampling are SQL, so they need one
with SQL enabled.

Sections now say where they differ: views have SDK methods on a
deployment, dropping a function needs a further setting, a local job
handle runs in-process and has no history, create_index and
refresh_column behave differently against a deployment, and the
overviews no longer call computed columns and materialized views
available everywhere.
2026-10-08 21:31:42 -07:00
Brendan Clement 05c7ba4889 docs: correct the labelled pages against the pinned deployment
Authorization wrote objects as namespace:db/ns and table:db/ns/table,
which the deployment rejects; it writes namespace:db:ns and
table:db:ns$table, nesting namespaces with $. The page now lists the
view, function and secret objects, CREATE_FUNCTION and CREATE_SECRET,
USAGE and UPDATE where they apply beyond tables, and what OPERATE and
USAGE on system allow. An a: subject names the principal holding a key
rather than a narrower grantee, creation falls back to the parent's
create privilege only for an unowned database or namespace, and a
listing page holds at most 100 entries.

The materialized view page said a deployment's refresh, options and
stable row id needs depend on version and configuration. Newer versions
rebuild the whole view on every create and refresh as a job, need no
stable row ids, take no refresh options, and have no REST route to create
one; earlier ones differed, and their views have to be recreated. The
page and the REFRESH reference now say so, without naming which version
a deployment runs.

Vector search no longer says it has no SQL form: some deployment
versions plan ORDER BY distance(...) LIMIT k as one.
2026-10-08 21:31:15 -07:00
Brendan Clement 03e573f442 docs: declare where four pages apply
Vector search, materialized view refresh, authentication and authorization
now say in their frontmatter whether LanceDB OSS and LanceDB Enterprise have
what they describe, with one sentence on how. The assembled site renders
that as each page's label, its sidebar tag and its comparison row, so the
hand-written badges and opening paragraphs on the two security pages go.

The availability was checked against the SDK and the Enterprise server:
authorization is enforced only on a deployment configured for it, and a
deployment accepts OAuth tokens only with an identity provider configured.
Sections now say that only the Rust SDK sets approx_mode, that a deployment
may search exactly minimum_nprobes partitions, and that the SDK refuses
with_no_data, like source_version, on a db:// connection.
2026-10-08 15:47:28 -07:00
Jack YeandBrendan Clement 47b7babc08 docs: bring the open-source documentation into this repository (#4122)
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).

This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.

Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.

**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.

**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.

Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.

**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.

The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.

---------

Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
2026-10-07 11:15:00 -07:00