Commit Graph
11 Commits
Author SHA1 Message Date
Brendan Clement e67f2b4b7b docs: say what the availability comparison covers
The comparison now lists pages across the documentation, grouped as in
the navigation. The page says what Yes, No and Varies mean, that pages
spanning several features are left out, and that the SQL row stands for
the SQL reference.
2026-10-08 21:31:43 -07:00
Brendan Clement 98d13f08c8 docs: declare where storage, training and integration pages apply
Storage configuration from the connection is embedded only, since a
deployment configures its own. Training data loading, PyTorch, embedding
functions, pandas, Polars and Pydantic work with local and deployment
tables, and the Hugging Face Hub integration is embedded only.

A local table cannot be pickled for DataLoader workers, so the PyTorch
page says to wrap it in a Permutation; a LazyFrame over a whole table,
to_pandas in the LangChain example and to_arrow in the DuckDB note are
local only; and query text is embedded in the client for both offerings.
2026-10-08 21:31:42 -07:00
Brendan Clement ac2db48af6 docs: declare where SQL and the SDKs apply
SQL is unavailable in the embedded library and varies on a deployment,
which serves it only when its Flight SQL endpoint is enabled and
reachable, and the SDK has to be told where that is. The page's opening
badge goes. The SDKs page declares both offerings.

Neither SDK aggregates or joins, so the parity tables no longer say they
do, and views can be created through the SDKs on a deployment. Pages that
said vector search has no SQL form now point to the version-dependent
SQL form, and COPY, which newer deployment versions refuse, is marked
as such in the reference.
2026-10-08 21:31:42 -07:00
Brendan Clement 6398ff3629 docs: declare where management, jobs, feature engineering and curation apply
Databases, namespaces, jobs and data evolution work in both offerings.
Materialized views, computed columns and column refresh work in the
embedded library and vary on a deployment, which has to enable them.
Views and functions exist only on a deployment that enables them, and
aggregation, joins, windowing and sampling are SQL, so they need one
with SQL enabled.

Sections now say where they differ: views have SDK methods on a
deployment, dropping a function needs a further setting, a local job
handle runs in-process and has no history, create_index and
refresh_column behave differently against a deployment, and the
overviews no longer call computed columns and materialized views
available everywhere.
2026-10-08 21:31:42 -07:00
Brendan Clement 6a1645116e docs: declare where ingestion and table pages apply
Creating, writing, updating, merging and deleting rows, managing tables,
branches and read consistency work in the embedded library and on a
deployment, and those pages now say so. The SQL forms they mention need
a deployment.

Where the offerings differ, the section says how: a deployment reuses an
existing table without comparing schemas, a merge insert can be refused
past a configured number of unindexed rows, space is reclaimed by
optimize() locally and by the deployment's compaction and configured
cleanup otherwise, renaming depends on the catalog, a deployment clones
shallowly only, drops all tables only locally, keeps freshness state per
table handle and defaults freshness to its own setting, and refuses to
delete a branch main depends on. table_exists works only on a namespace
connection.

Guarantees and limits span both offerings, so they carry section notes
rather than a label: reads, retention, index freshness and compaction
differ on a deployment, and the name and job limits are a deployment's.
2026-10-08 21:31:42 -07:00
Brendan Clement 027597d36d docs: declare where indexing and maintenance apply
Indexes are built in the embedded library and on a deployment, which
builds them in the background, keeps most of them current, and builds
only IVF_FLAT, IVF_PQ, IVF_RQ and IVF_HNSW_SQ vector indexes. Compaction
runs on both, by optimize() locally and in the background on a
deployment; cleanup on a deployment runs only where its configuration
enables it, so it varies; background jobs exist only on a deployment.

Sections that described one offering as if it were both now name it:
optimize() is embedded only, the brute-force limit and automatic IVF_PQ
are deployment settings, FM indexes get no background updates, a
remainder below the deployment's threshold can keep wait_for_index from
finishing, GPU builds on a deployment are IVF_PQ with a configured GPU
count, list_indices on a deployment omits num_unindexed_rows, and the
background jobs page no longer says every other job is one you start.
2026-10-08 21:31:25 -07:00
Brendan Clement e85f1b4434 docs: declare where the search pages apply
Search, full-text search, hybrid search, filtering, query plans and
reranking all work in the embedded library and against a deployment, and
now say so. Where the two differ, the section says how: a deployment
returns a query's whole result in one response, the synchronous Python
remote table refuses to_pandas, to_arrow and to_lance, probe widening
depends on the deployment's version, full-text indexes build in the
background and are kept current without optimize(), distributed scoring
can differ slightly, model files have to be present on the deployment,
plans carry deployment operators, and hybrid explain_plan is Python only.
Reranking runs in the application either way.
2026-10-08 21:31:25 -07:00
Brendan Clement 05c7ba4889 docs: correct the labelled pages against the pinned deployment
Authorization wrote objects as namespace:db/ns and table:db/ns/table,
which the deployment rejects; it writes namespace:db:ns and
table:db:ns$table, nesting namespaces with $. The page now lists the
view, function and secret objects, CREATE_FUNCTION and CREATE_SECRET,
USAGE and UPDATE where they apply beyond tables, and what OPERATE and
USAGE on system allow. An a: subject names the principal holding a key
rather than a narrower grantee, creation falls back to the parent's
create privilege only for an unowned database or namespace, and a
listing page holds at most 100 entries.

The materialized view page said a deployment's refresh, options and
stable row id needs depend on version and configuration. Newer versions
rebuild the whole view on every create and refresh as a job, need no
stable row ids, take no refresh options, and have no REST route to create
one; earlier ones differed, and their views have to be recreated. The
page and the REFRESH reference now say so, without naming which version
a deployment runs.

Vector search no longer says it has no SQL form: some deployment
versions plan ORDER BY distance(...) LIMIT k as one.
2026-10-08 21:31:15 -07:00
Brendan Clement f228295c34 docs: add the OSS and Enterprise and connection pages
basics/offerings explains the two offerings and holds the comparison the
assembled site generates from the pages' availability. basics/connect
shows local, object storage and Enterprise connections with the existing
tested connection snippets. Both follow the landing page in Getting
Started.
2026-10-08 15:59:35 -07:00
Brendan Clement 03e573f442 docs: declare where four pages apply
Vector search, materialized view refresh, authentication and authorization
now say in their frontmatter whether LanceDB OSS and LanceDB Enterprise have
what they describe, with one sentence on how. The assembled site renders
that as each page's label, its sidebar tag and its comparison row, so the
hand-written badges and opening paragraphs on the two security pages go.

The availability was checked against the SDK and the Enterprise server:
authorization is enforced only on a deployment configured for it, and a
deployment accepts OAuth tokens only with an identity provider configured.
Sections now say that only the Rust SDK sets approx_mode, that a deployment
may search exactly minimum_nprobes partitions, and that the SDK refuses
with_no_data, like source_version, on a db:// connection.
2026-10-08 15:47:28 -07:00
Jack YeandBrendan Clement 47b7babc08 docs: bring the open-source documentation into this repository (#4122)
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).

This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.

Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.

**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.

**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.

Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.

**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.

The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.

---------

Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
2026-10-07 11:15:00 -07:00