From 1db0d3ca5f57303419a1522e2527a285e6254eb2 Mon Sep 17 00:00:00 2001 From: Brendan Clement Date: Thu, 5 Nov 2026 00:13:02 -0800 Subject: [PATCH] docs: label early access features and hide ones not ready to document (#4855) Labels early access features and hides features we aren't documenting yet. **Early access label:** Functions (4 pages, plus the GPU refresh, checkpointing and adaptive execution pages), Secrets (3 pages), Tiered Index Cache, Blob V2, and merge insert, which is labelled for Enterprise only. Two sidebar titles are shorter so the tag fits. **Hidden:** branching, JSON columns and geospatial. - The JSON functions page and the agent branch-experiments tutorial go in `.mintignore`. - `tables/branching` stays reachable but out of the nav, as #4859 left it, because SDK error messages link to it. - Other pages' mentions of these features become draft comments, so they're easy to restore. **Secrets page:** drops references to manual KMS setup, which the control plane handles. It keeps the encryption-at-rest facts, states the Azure value-size limit inline, and points a 501 error to support. --- docs/web/.mintignore | 7 +++++++ docs/web/basics/embedded-vs-enterprise.mdx | 6 +++++- docs/web/basics/naming.mdx | 14 +++++++++++--- docs/web/curation/create-function.mdx | 3 +++ docs/web/curation/gpu-function-execution.mdx | 5 ++++- docs/web/curation/vend-secret.mdx | 3 +++ docs/web/docs.json | 4 +--- docs/web/enterprise/adaptive-execution.mdx | 3 +++ docs/web/enterprise/checkpointing.mdx | 3 +++ docs/web/enterprise/gpu-refresh-execution.mdx | 3 +++ docs/web/enterprise/tiered-index-cache.mdx | 5 ++++- docs/web/index.mdx | 6 +++++- docs/web/ingestion/merge-insert.mdx | 2 ++ docs/web/maintenance/cleanup.mdx | 14 ++++++++++---- docs/web/management/function.mdx | 3 +++ docs/web/management/secret.mdx | 15 ++++++++------- docs/web/management/table.mdx | 18 ++++++++++++------ docs/web/search/realtime-read.mdx | 4 ++-- docs/web/sql/data-types.mdx | 10 +++++++++- docs/web/sql/ddl/function.mdx | 3 +++ docs/web/sql/ddl/secret.mdx | 3 ++- docs/web/sql/dql/functions.mdx | 8 ++++++-- docs/web/sql/dql/scalar-functions.mdx | 10 +++++++++- docs/web/tutorials/index.mdx | 2 +- 24 files changed, 119 insertions(+), 35 deletions(-) diff --git a/docs/web/.mintignore b/docs/web/.mintignore index 388087951..8fea3d17d 100644 --- a/docs/web/.mintignore +++ b/docs/web/.mintignore @@ -1 +1,8 @@ tools/cli.mdx +# Branching is Beta; not documented until early access. tables/branching.mdx +# stays reachable (out of the nav only): SDK error messages link to it. +# To restore: docs.json Tutorials > Feature Engineering, after tutorials/feature-engineering/materialized-views. +agent-branch-experiments.mdx +# JSON columns are Beta; not documented until early access. +# To restore: docs.json Documentation > SQL Reference > DQL, after sql/dql/vector-distance. +sql/dql/json-functions.mdx diff --git a/docs/web/basics/embedded-vs-enterprise.mdx b/docs/web/basics/embedded-vs-enterprise.mdx index c11324300..da58acb7d 100644 --- a/docs/web/basics/embedded-vs-enterprise.mdx +++ b/docs/web/basics/embedded-vs-enterprise.mdx @@ -78,7 +78,6 @@ file format, so moving data from one edition to the other does not require a dat | | [Database](/management/database) | ✅ | ✅ | | | [Namespace](/management/namespace) | ✅ | ✅ | | | [Table](/management/table) | ✅ | ✅ | -| | [Branches](/tables/branching) | ✅ | ✅ | | | [View](/management/view) | ❌ | ✅ | | | [Materialized view](/management/materialized-view) | ❌ | ✅ | | | [Secret](/management/secret) | ❌ | ✅ | @@ -104,6 +103,11 @@ file format, so moving data from one edition to the other does not require a dat | | [OpenTelemetry](/deployment/opentelemetry) | ✅ | ✅ | | | [Kubernetes](/enterprise/deployment/kubernetes) | ❌ | ✅ | +{/* +Draft until branching reaches early access; restore to the Management rows. +| | [Branches](/tables/branching) | ✅ | ✅ | +*/} + ## Usage differences between Enterprise and Embedded {#usage-differences-between-enterprise-and-oss} The [quickstart](/quickstart) guide shows both LanceDB Embedded connections and Enterprise `db://...` diff --git a/docs/web/basics/naming.mdx b/docs/web/basics/naming.mdx index 165840218..b00f50bf4 100644 --- a/docs/web/basics/naming.mdx +++ b/docs/web/basics/naming.mdx @@ -10,7 +10,7 @@ keywords: ["names", "naming", "table name", "namespace", "identifier", "limits"] -Databases, namespaces, tables, views and branches all have names. Choose names +Databases, namespaces, tables and views all have names.{/* Draft until branching reaches early access: restore "Databases, namespaces, tables, views and branches all have names." */} Choose names that describe their purpose, such as `documents`, `prod` or `training_2026`. Consistent names make data easier to find and application code easier to read. @@ -21,10 +21,14 @@ Use the following rules for names that work across LanceDB Embedded and LanceDB | Table, materialized view, view | ASCII letters and digits, `-`, `_`, `.` | 1 to 128 characters | | Namespace component | ASCII letters and digits, `-`, `_`, `.` | 1 to 64 characters; at most 10 components in a path | | Database | ASCII letters and digits, `-`, `_`, `.`, with `/` between components | 1 to 64 characters for the whole name | -| Branch | Avoid `$` and `:`; use ASCII letters, digits, `-` and `_` for portability | Keep to 1 to 64 characters | | Secret | ASCII letters and digits, `-`, `_`, `.` | 1 to 128 characters | | Function | ASCII letters and digits, `_` | 1 to 128 characters | +{/* +Draft until branching reaches early access; restore after the Database row. +| Branch | Avoid `$` and `:`; use ASCII letters, digits, `-` and `_` for portability | Keep to 1 to 64 characters | +*/} + - Names and individual namespace or database components must not be empty. - Table, view, namespace and database components cannot be `.` or `..`. - Secret names cannot be `.` or `..`. @@ -60,5 +64,9 @@ The empty path `[]` identifies the root namespace. ## Learn more {#next} - [Namespaces](/management/namespace): create namespaces and address tables. -- [Branches](/tables/branching): name and manage separate lines of table history. - [Limits](/basics/limits): check other constraints before designing your schema. + +{/* +Draft until branching reaches early access; restore after Namespaces. +- [Branches](/tables/branching): name and manage separate lines of table history. +*/} diff --git a/docs/web/curation/create-function.mdx b/docs/web/curation/create-function.mdx index 232cf79dc..a2751cf8f 100644 --- a/docs/web/curation/create-function.mdx +++ b/docs/web/curation/create-function.mdx @@ -1,5 +1,6 @@ --- title: "Create a Function" +tag: "Early access" sidebarTitle: "Create Function" description: "Register Python as a column: CPU, GPU, and remote-service Functions, called once per row or once per batch." icon: code @@ -9,6 +10,8 @@ availability: enterprise: available --- +Early access + Some curation steps cannot be written as a SQL expression: an embedding model, an image decode, a call to an external service. A Function is a Python callable registered with the database, versioned, and then applied as diff --git a/docs/web/curation/gpu-function-execution.mdx b/docs/web/curation/gpu-function-execution.mdx index f769faef4..e7f6f0f6a 100644 --- a/docs/web/curation/gpu-function-execution.mdx +++ b/docs/web/curation/gpu-function-execution.mdx @@ -1,6 +1,7 @@ --- title: "GPU Function Execution" -sidebarTitle: "GPU Function Execution" +tag: "Early access" +sidebarTitle: "GPU Functions" description: "How a GPU Function's worker gets scheduled onto a GPU-capable node, and what happens when none is free." icon: microchip keywords: ["gpu", "gpu class", "scheduling", "function", "udf", "kubernetes", "node selector"] @@ -9,6 +10,8 @@ availability: enterprise: available --- +Early access + [Registering a GPU Function](/curation/create-function#gpu) sets `gpu=True` and an optional `gpu_class`; this page covers what happens after that, when a job actually needs a worker. An operator configures one or more named diff --git a/docs/web/curation/vend-secret.mdx b/docs/web/curation/vend-secret.mdx index 6a16fcdf0..801f16436 100644 --- a/docs/web/curation/vend-secret.mdx +++ b/docs/web/curation/vend-secret.mdx @@ -1,5 +1,6 @@ --- title: "Vend a Secret to a Function" +tag: "Early access" sidebarTitle: "Vend Secret" description: "Bind a Secret to a Function by reference, and how its value reaches running code during SQL evaluation and refresh." icon: key @@ -9,6 +10,8 @@ availability: enterprise: available --- +Early access + A Function that calls an external service needs a credential without putting it in the function's source -- see [Create a Function](/curation/create-function#remote) for how to register diff --git a/docs/web/docs.json b/docs/web/docs.json index bdc849f63..58ad78a52 100644 --- a/docs/web/docs.json +++ b/docs/web/docs.json @@ -590,7 +590,6 @@ "sql/dql/dedupe", "sql/dql/duplicate-pairs", "sql/dql/vector-distance", - "sql/dql/json-functions", "sql/dql/blob-descriptor", "sql/dql/generate-series", "sql/dql/explain" @@ -708,8 +707,7 @@ "group": "Feature Engineering", "pages": [ "tutorials/feature-engineering/feature-engineering-101", - "tutorials/feature-engineering/materialized-views", - "agent-branch-experiments" + "tutorials/feature-engineering/materialized-views" ] }, { diff --git a/docs/web/enterprise/adaptive-execution.mdx b/docs/web/enterprise/adaptive-execution.mdx index 3584cc191..0c8c20b86 100644 --- a/docs/web/enterprise/adaptive-execution.mdx +++ b/docs/web/enterprise/adaptive-execution.mdx @@ -1,5 +1,6 @@ --- title: "Adaptive Execution" +tag: "Early access" sidebarTitle: "Adaptive Execution" description: "How a Function refresh's part size, resource class, worker count and checkpoint frequency adjust themselves to observed behavior." icon: "gauge-high" @@ -8,6 +9,8 @@ availability: enterprise: available --- +Early access + When a [computed column](/feature-engineering/add-computed-column) or [materialized view](/feature-engineering/materialized-views) is filled by a registered [Function](/curation/create-function), LanceDB Enterprise diff --git a/docs/web/enterprise/checkpointing.mdx b/docs/web/enterprise/checkpointing.mdx index 1569071d0..ca6eab814 100644 --- a/docs/web/enterprise/checkpointing.mdx +++ b/docs/web/enterprise/checkpointing.mdx @@ -1,5 +1,6 @@ --- title: "Function Refresh Checkpointing" +tag: "Early access" sidebarTitle: "Checkpointing" description: "How a Function refresh's progress survives a worker failure or restart, part by part." icon: "floppy-disk" @@ -9,6 +10,8 @@ availability: enterprise: available --- +Early access + This page is about checkpointing a [Function refresh](/enterprise/adaptive-execution) specifically -- not [PyTorch training checkpointing](/training/checkpointing), a different, unrelated mechanism covered elsewhere. diff --git a/docs/web/enterprise/gpu-refresh-execution.mdx b/docs/web/enterprise/gpu-refresh-execution.mdx index b9c7e6acc..7fcacf4b5 100644 --- a/docs/web/enterprise/gpu-refresh-execution.mdx +++ b/docs/web/enterprise/gpu-refresh-execution.mdx @@ -1,5 +1,6 @@ --- title: "GPU Refresh Execution" +tag: "Early access" sidebarTitle: "GPU Refresh Execution" description: "A computed column or materialized view filled by a GPU Function schedules its refresh workers the same way GPU Function execution does in Data Curation." icon: microchip @@ -9,6 +10,8 @@ availability: enterprise: available --- +Early access + A [computed column](/feature-engineering/add-computed-column) or [materialized view](/feature-engineering/materialized-views) filled by a registered [Function](/curation/create-function) with `gpu=True` refreshes diff --git a/docs/web/enterprise/tiered-index-cache.mdx b/docs/web/enterprise/tiered-index-cache.mdx index 439c52df2..f807f04ae 100644 --- a/docs/web/enterprise/tiered-index-cache.mdx +++ b/docs/web/enterprise/tiered-index-cache.mdx @@ -1,6 +1,7 @@ --- title: "Tiered Index Cache" -sidebarTitle: "Tiered Index Cache" +tag: "Early access" +sidebarTitle: "Index Cache" description: "How LanceDB Enterprise caches deserialized index structures across memory and local disk." icon: "layer-group" availability: @@ -8,6 +9,8 @@ availability: enterprise: available --- +Early access + The tiered index cache holds deserialized vector, full-text and scalar index structures in memory, backed by a local-disk tier beneath it. When memory fills, the cache serializes evicted entries and spills them to disk diff --git a/docs/web/index.mdx b/docs/web/index.mdx index 8b290b97a..7b6bc6eed 100644 --- a/docs/web/index.mdx +++ b/docs/web/index.mdx @@ -23,7 +23,7 @@ time stitching infrastructure together and more time improving datasets, testing ## Accelerate the data flywheel for model research {#accelerate-the-data-flywheel-for-model-research} -Keep your AI data in one table, and give each agent-driven experiment its own isolated branch. +Keep your AI data in one table.{/* Draft until branching reaches early access: restore ", and give each agent-driven experiment its own isolated branch" before the period. */} Find the failure, fix the data, and retrain. @@ -31,11 +31,15 @@ Find the failure, fix the data, and retrain. Start from one failed grasp or evaluation example. [Search](/search/) multimodal data, metadata, and embeddings together, retrieve similar examples, and [curate](/curation) them into a targeted training subset. + {/* + Draft until branching reaches early access. + Don't let an agent silently corrupt your data and ruin a training run. Give each agent its own [branch](/tables/branching), a writable fork of the table, to experiment on, and review the changes before they reach `main`. See [Run experiments on branches](/agent-branch-experiments). + */} Keep episodes, frames, metadata, features, and embeddings together in one schema-enforced table, without pipelines to keep them in sync. Table [versions](/management/table#versioning) let you trace each training diff --git a/docs/web/ingestion/merge-insert.mdx b/docs/web/ingestion/merge-insert.mdx index d0ecab8b9..baf241b30 100644 --- a/docs/web/ingestion/merge-insert.mdx +++ b/docs/web/ingestion/merge-insert.mdx @@ -30,6 +30,8 @@ import { import { PyWriteWorkflow, TsWriteWorkflowBody, TsWriteWorkflowTail, RsWriteWorkflow } from '/snippets/ingestion_examples.mdx'; +Early access in Enterprise + Use merge insert when incoming records should be compared with existing rows. Combine update and insert policies for an **upsert**. Use [append](/ingestion/insert) when every incoming row should simply be added. diff --git a/docs/web/maintenance/cleanup.mdx b/docs/web/maintenance/cleanup.mdx index c7fc2cabe..21ba079f9 100644 --- a/docs/web/maintenance/cleanup.mdx +++ b/docs/web/maintenance/cleanup.mdx @@ -37,7 +37,7 @@ retention period comes from its configuration instead. The latest version is never removed, whatever you pass. `timedelta(0)` makes every older version eligible for removal, but cleanup still keeps tagged -versions and the files branches still read, as the warning below describes. +versions{/* Draft until branching reaches early access: restore " and the files branches still read" */}, as the warning below describes. Pruning is what makes [time travel](/management/table#versioning) finite: an @@ -47,12 +47,18 @@ whatever its age, when it is: - the **latest** version; - a **tagged** version. +{/* +Draft until branching reaches early access. + Cleanup also keeps the files a **branch** still reads, and the version the branch was forked from while the branch's remaining versions read the parent's files. That does not keep the fork for good: once the branch has replaced the data it inherited and its own older versions are pruned, cleanup prunes the -fork version like any other. To keep a version recoverable, tag it, subject to -the policy below, or keep the window wide enough to cover it. +fork version like any other. +*/} + +To keep a version recoverable, tag it, subject to the policy below, or keep +the window wide enough to cover it. Under the default policy, a tag can also stop cleanup. When a tagged version is older than the window and is not the latest, cleanup refuses to run: it raises @@ -75,7 +81,7 @@ need before running this, not after. [Managed blobs](/management/table#managed-blobs) store out-of-line payloads under the table's `_blobs/` directory. Cleanup follows the references from -retained snapshots, including protected tags and branches, before selecting +retained snapshots, including protected tags{/* Draft until branching reaches early access: restore " and branches" */}, before selecting objects for deletion. Replacing a blob or deleting a row makes its old payload eligible only after no retained snapshot references it. diff --git a/docs/web/management/function.mdx b/docs/web/management/function.mdx index 38a346c18..3a4f0736c 100644 --- a/docs/web/management/function.mdx +++ b/docs/web/management/function.mdx @@ -1,5 +1,6 @@ --- title: "Functions" +tag: "Early access" sidebarTitle: "Function" description: "Register a Python function once and apply it as a column, for work SQL cannot express." icon: code @@ -23,6 +24,8 @@ import { } from '/snippets/function.mdx'; import { PyConnectEnterprisePlaceholders as FunctionConnect } from '/snippets/connection.mdx'; +Early access + Some columns cannot be written as a SQL expression: an embedding, a model call, an image decode. A function is a Python callable registered with the database, versioned, and then applied as a column. The code runs where the data is diff --git a/docs/web/management/secret.mdx b/docs/web/management/secret.mdx index 0359ce6a3..b8a37fb51 100644 --- a/docs/web/management/secret.mdx +++ b/docs/web/management/secret.mdx @@ -1,5 +1,6 @@ --- title: "Secret" +tag: "Early access" sidebarTitle: "Secret" description: "Create, rotate, list, describe and drop a named credential that Functions bind by reference." icon: key @@ -9,6 +10,8 @@ availability: enterprise: available --- +Early access + A **Secret** is a catalog object, like a table or a namespace: a named credential, such as a model provider's API key, stored once and addressed by name. A [Function](/curation/create-function) binds a Secret by reference @@ -109,12 +112,10 @@ Every Secret's value is encrypted directly with your deployment's key in AWS KMS, Google Cloud KMS, or Azure Key Vault -- there is no separate application data key to manage. Storage, backups and replicas hold only ciphertext. Decryption goes through your KMS, which records it in your cloud's audit -log. A deployment without a KMS key does not offer Secrets; see -[KMS integration](/enterprise/deployment/kms). +log. BYOC You are responsible for the network egress -policy for the jobs that run Functions, and for scoping the Function -identity's cloud role to the KMS key so it can decrypt but not encrypt. +policy for the jobs that run Functions. ### Redaction {#redaction} @@ -131,7 +132,7 @@ that holds that credential. | Item | Limit | | --- | --- | | Secret name | 1--128 characters from `A-Z a-z 0-9 _ . -`. `$` is reserved as the namespace delimiter. | -| Value | One nonempty UTF-8 string, at most 4,096 bytes. Azure Key Vault RSA keys allow less -- see [KMS integration](/enterprise/deployment/kms). | +| Value | One nonempty UTF-8 string, at most 4,096 bytes. On Azure, at most 158, 286 or 414 bytes for an RSA-2048, RSA-3072 or RSA-4096 key. | | Bindings per Function | 16 | | Environment variable name | `[A-Za-z_][A-Za-z0-9_]*`, and not [reserved](#reserved) | | Variables per Function | Each bound once. Two variables may bind the same Secret. | @@ -157,8 +158,8 @@ refreshing one: the Secret was never created in that namespace, was dropped, or its name is misspelled in the binding. Create it, or register a new Function version with the right name. -**501 from a Secret API call**: the deployment has no KMS key configured, so -Secrets are disabled. See [KMS integration](/enterprise/deployment/kms). +**501 from a Secret API call**: Secrets aren't enabled for this deployment. +Contact [LanceDB support](/deployment/support). ## Privileges {#privileges} diff --git a/docs/web/management/table.mdx b/docs/web/management/table.mdx index 14ff57cf4..c229e13f0 100644 --- a/docs/web/management/table.mdx +++ b/docs/web/management/table.mdx @@ -143,8 +143,8 @@ import { A table stores data in rows and columns and keeps a history of committed versions. Queries read table data, writes update it, and indexes accelerate -searches over its columns. Branches let you make changes separately from the -main table history. +searches over its columns.{/* Draft until branching reaches early access: restore "Branches let you make changes separately from the +main table history." */} This page covers table management, data types, schema changes and version history. For loading records, see [Ingestion](/ingestion). For changes to @@ -161,7 +161,11 @@ In LanceDB Enterprise, tables are also created and dropped in SQL with [`CREATE | **Columns** | including vectors, blobs, and [computed](/feature-engineering/add-computed-column) columns | | **Versions** | every write commits one, so reads can [travel back](/management/table#versioning) | | **Indexes** | vector, full-text and scalar, built over its columns | + +{/* +Draft until branching reaches early access. | **Branches** | isolated lines of change you can [merge or discard](/tables/branching) | +*/} ## Create a table {#create} @@ -326,7 +330,7 @@ as it is to the format. | String and binary | `utf8`, `large_utf8`, `binary`, `large_binary`, `fixed_size_binary(n)` | | Date and time | `date32`, `date64`, `time32`, `time64`, `timestamp`, `duration` | | Nested | `struct`, `list`, `large_list`, `fixed_size_list`, `map` | -| Lance extensions | vectors, [JSON](/sql/data-types#json), [blobs](#multimodal), bfloat16 | +| Lance extensions | vectors, [blobs](#multimodal), bfloat16{/* Draft until JSON columns reach early access: restore "[JSON](/sql/data-types#json), " after "vectors, " */} | A vector column is a `fixed_size_list`, an ordinary Arrow type, which is why it is selectable and joinable but why arithmetic over it is list @@ -345,9 +349,7 @@ file, up to 64 KiB by default, and larger values are stored outside it. ### Blob columns {#multimodal-blob-v2} - -Blob V2 is in early access. - +Early access A `blob` column keeps metadata separate from the payload bytes, so queries can filter rows without loading the blobs. @@ -1490,6 +1492,9 @@ Deleting a tag only removes the label, not the version it points to. After deletion, the underlying table version becomes eligible for cleanup again. +{/* +Draft until branching reaches early access. + ### Branches {#versioning-branches} Beyond linear history, LanceDB also supports **branches**: isolated, writable @@ -1499,6 +1504,7 @@ writable history, making it ideal for experiments, backfills, and migrations that you want to keep separate from production reads on `main`. Branches are covered in their own guide: see [Branches](/tables/branching). +*/} ### Delete Data From the Table {#versioning-delete-data-from-the-table} diff --git a/docs/web/search/realtime-read.mdx b/docs/web/search/realtime-read.mdx index cebfd6d77..e586a1eac 100644 --- a/docs/web/search/realtime-read.mdx +++ b/docs/web/search/realtime-read.mdx @@ -70,8 +70,8 @@ returning a result that omits un-compacted rows. ## Limitations {#limitations} -Support for all table operations on RTE tables is coming soon. Time travel, -branching, `count_rows`, row IDs (`with_row_id`), and `TABLESAMPLE` are not yet +Support for all table operations on RTE tables is coming soon. Time travel,{/* Draft until branching reaches early access: restore " branching," */} +`count_rows`, row IDs (`with_row_id`), and `TABLESAMPLE` are not yet available. diff --git a/docs/web/sql/data-types.mdx b/docs/web/sql/data-types.mdx index 651d04d29..ef5dfcab7 100644 --- a/docs/web/sql/data-types.mdx +++ b/docs/web/sql/data-types.mdx @@ -3,7 +3,7 @@ title: "SQL data types" sidebarTitle: "Data Types" description: "How Lance columns appear to SQL, including the types SQL has no name for." icon: shapes -keywords: ["sql types", "arrow", "cast", "pseudo-columns", "vector", "json"] +keywords: ["sql types", "arrow", "cast", "pseudo-columns", "vector"] availability: embedded: unavailable enterprise: available @@ -84,6 +84,9 @@ calculations. SQL vector search depends on the Enterprise server version and que plan; see [SQL](/basics/sql#no-vector-sql). +{/* +Draft until JSON columns reach early access. + ### JSON {#json} Stored as JSONB in `large_binary` with the `lance.json` extension name. It reads @@ -98,6 +101,7 @@ WHERE json_get_int(data, 'status') = 200 `json_get`, its typed variants, and `json_extract` are **index-accelerated** by a JSON scalar index. See [JSON functions](/sql/dql/json-functions). +*/} ### bfloat16 {#bfloat16} @@ -105,6 +109,9 @@ Sixteen-bit floats in the bfloat16 layout, stored as `fixed_size_binary(2)`. Useful as a vector element type when halving the memory matters more than the precision does. +{/* +Draft: geospatial queries and indexes are unsupported in LanceDB Enterprise. + ## Geospatial {#geospatial} @@ -118,6 +125,7 @@ spatial index is gated the same way. Geometry can still be stored (it is binary, and binary columns work), but nothing interprets it. Filter on a bounding box held in ordinary numeric columns if you need spatial predicates today. +*/} ## Pseudo-columns {#pseudo-columns} diff --git a/docs/web/sql/ddl/function.mdx b/docs/web/sql/ddl/function.mdx index 5dbdb58ae..8cd45f8bc 100644 --- a/docs/web/sql/ddl/function.mdx +++ b/docs/web/sql/ddl/function.mdx @@ -1,5 +1,6 @@ --- title: "Function" +tag: "Early access" sidebarTitle: "Function" description: "Create, describe, alter, list, and drop registered functions with SQL." icon: function @@ -9,6 +10,8 @@ availability: enterprise: available --- +Early access + [SQL runs in LanceDB Enterprise.](/sql) ## CREATE FUNCTION {#create} diff --git a/docs/web/sql/ddl/secret.mdx b/docs/web/sql/ddl/secret.mdx index 33ba009da..6c6e3c4c6 100644 --- a/docs/web/sql/ddl/secret.mdx +++ b/docs/web/sql/ddl/secret.mdx @@ -1,12 +1,13 @@ --- title: "Secret" +tag: "Early access" sidebarTitle: "Secret" description: "CREATE, ALTER, DROP, SHOW and DESCRIBE SECRET, and granting privileges on Secrets." icon: key keywords: ["secret", "create secret", "alter secret", "drop secret", "show secrets", "describe secret", "grant", "sql"] --- -Enterprise [SQL runs in LanceDB Enterprise.](/sql) +Early access Enterprise [SQL runs in LanceDB Enterprise.](/sql) A [Secret](/management/secret) is a named credential that Functions bind by reference. These statements store, rotate, list, describe and drop Secrets. diff --git a/docs/web/sql/dql/functions.mdx b/docs/web/sql/dql/functions.mdx index 945abbe03..0ea53a508 100644 --- a/docs/web/sql/dql/functions.mdx +++ b/docs/web/sql/dql/functions.mdx @@ -15,7 +15,7 @@ plus what Lance adds. | | | | --- | --- | -| [Scalar functions](/sql/dql/scalar-functions) | per row: math, string, regex, date/time, array, struct, JSON | +| [Scalar functions](/sql/dql/scalar-functions) | per row: math, string, regex, date/time, array, struct{/* Draft until JSON columns reach early access: restore ", JSON" */} | | [Aggregate functions](/sql/dql/aggregation#functions) | per group: `count`, `sum`, statistical, approximate | | [Window functions](/sql/dql/window#functions) | per window: ranking, offset, running aggregates | | [Table functions](#table-functions) | produce a relation rather than a value | @@ -45,13 +45,17 @@ see [Data types](/sql/data-types#pseudo-columns). ## What Lance accelerates {#accelerated} -Two function families use an index rather than scanning, which is the difference +`contains_tokens` uses an index rather than scanning, which is the difference between a fast predicate and a slow one: | | Index used | | --- | --- | | [`contains_tokens`](/sql/dql/contains-tokens) | full-text index | + +{/* +Draft until JSON columns reach early access; restore "Two function families" above. | [`json_get`, typed variants, `json_extract`](/sql/dql/json-functions) | JSON scalar index | +*/} Confirm with [`EXPLAIN`](/sql/dql/explain): an accelerated predicate shows an index node rather than a `LanceScan` followed by a filter. diff --git a/docs/web/sql/dql/scalar-functions.mdx b/docs/web/sql/dql/scalar-functions.mdx index b0910c01a..9bd2057b0 100644 --- a/docs/web/sql/dql/scalar-functions.mdx +++ b/docs/web/sql/dql/scalar-functions.mdx @@ -3,7 +3,7 @@ title: "Scalar functions" sidebarTitle: "Scalar Functions" description: "The per-row function set: math, string, regex, date/time, array, struct, and what Lance adds." icon: calculator -keywords: ["scalar functions", "string functions", "math functions", "regex", "array functions", "json"] +keywords: ["scalar functions", "string functions", "math functions", "regex", "array functions"] availability: embedded: unavailable enterprise: available @@ -133,7 +133,14 @@ reference page: | [`contains_tokens`](/sql/dql/contains-tokens) | index-accelerated keyword predicate, usable in `WHERE` | | [`ivf_partition`](/sql/dql/ivf-partition) | assign each row to its trained IVF index partition | | [`blob_descriptor`](/sql/dql/blob-descriptor) | a BlobV2 column's payload location, without reading it | + +{/* +Draft until JSON columns reach early access. | [JSON accessors](/sql/dql/json-functions) | `json_get` and its typed variants, `json_extract`, and more | +*/} + +{/* +Draft: geospatial queries and indexes are unsupported in LanceDB Enterprise. ### Geospatial {#geo} @@ -143,6 +150,7 @@ Lance defines `st_*` functions on GeoArrow, but they sit behind a build feature feature rather than an unknown-function error, and the spatial index is gated the same way. +*/} ## Additional scalar functions {#escape-hatch} diff --git a/docs/web/tutorials/index.mdx b/docs/web/tutorials/index.mdx index 6ada57c9a..b9aea29a2 100644 --- a/docs/web/tutorials/index.mdx +++ b/docs/web/tutorials/index.mdx @@ -21,7 +21,7 @@ are runnable in Colab; the rest are written up here. Retrieval-augmented generation and agent applications. - Deriving columns, materialized views, experimenting on branches. + Deriving columns and materialized views.{/* Draft until branching reaches early access: restore "Deriving columns, materialized views, experimenting on branches." */} Curating data and training models directly on a table.