mirror of
https://github.com/lancedb/lancedb.git
synced 2026-08-18 12:08:35 +00:00
docs(python): fill gaps in the Python API reference (#3746)
`docs/src/python/python.md` is the whole Python API reference, but it is maintained by hand and had drifted from the public API. Anything not listed there simply doesn't get rendered, so a number of public, documented, tested APIs were invisible to users — most notably branch management, where `diff` and `merge` live. I audited every public symbol reachable from `lancedb` and its subpackages against the `:::` directives on the page. This adds the missing ones: - **Branching** — `Branches`, `AsyncBranches` (`list` / `create` / `checkout` / `delete` / `diff` / `merge`) - **Tables** — `TableStatistics` (returned by `Table.stats()`; the fragment-level stats classes were already listed) - **Full text queries** — `FullTextQuery`, `MatchQuery`, `PhraseQuery`, `BoostQuery`, `MultiMatchQuery`, `BooleanQuery`, `FullTextOperator`, `Occur` - **Querying** — `LanceEmptyQueryBuilder`, `LanceTakeQueryBuilder`, `AsyncTakeQuery` - **Indices** — `Fm` (the FM-index for substring search), `IndexConfig` - **Blobs** — `blob`, `BlobType`, `BlobFile` - **Namespaces** — `connect_namespace`, `connect_namespace_async`, and both namespace connection classes - **Remote config** — `TlsConfig`, `HeaderProvider`, `OAuthConfig`, `OAuthFlowType` - **Rerankers** — the `Reranker` base class plus `JinaReranker`, `RRFReranker`, `MRRReranker`, `AnswerdotaiRerankers`, `VoyageAIReranker`, `WatsonxReranker` (5 of 12 were listed) - **Embeddings** — `get_registry`, `register`, and the 14 embedding functions that were missing (3 of 17 were listed) - **PyTorch** — `StreamingDataset` and the permutation API it is built on - **Misc** — `Session`, `tokenize`, `FtsToken`, `pydantic.Vector`, `pydantic.MultiVector`, `instrument_lancedb_metrics`, and the two exception types It also repairs cross-references in docstrings that no longer resolve: links into guide pages that have since moved to lancedb.com (`querying-an-ann-index`, `experimental-full-text-search`), `lance.dataset` references with no inventory behind them, and the relative targets `[Table](Table)` and `[PyArrow Table](pyarrow.Table)`. Deliberately left out: concrete implementation classes reached through their abstract base (`LanceTable`, `LanceDBConnection`, `RemoteDBConnection`), query base classes already covered by `inherited_members: true`, and internal plumbing such as `FullTextSearchQuery` and `ColumnOrdering`. ## Testing The docs job only runs on pushes to `main`, so I built the site locally and compared against a build of `upstream/main`: every added entry resolves, and no symbol that was rendered before stopped being rendered when the four packages moved to automodule. `mkdocs build --strict` exits 0 on this branch, against 61 warnings on `main`. ## Also in this PR `lancedb.index`, `lancedb.embeddings`, `lancedb.remote` and `lancedb.rerankers` are now rendered by a single mkdocstrings directive each, driven by the module's `__all__`, rather than a hand-maintained list. These four are where most of the drift was, and `__all__` is harder to forget than a docs page. `lancedb.embeddings` had no `__all__`; without one mkdocstrings renders no members at all for a re-export package, so one is added. AGENTS.md gains a section on how the page is wired up and how to build the docs locally. Rendering all that code for the first time surfaced ~100 more build warnings, which would have made #3707 (turning on `mkdocs build --strict`) harder to land, so the warning backlog is cleared here too. 97 of the 158 warnings were one systematic false positive — griffe cannot see the generated `__init__` of a pydantic dataclass, so every documented parameter looks unknown — switched off via `warn_unknown_params`. The remaining 61 came from 15 docstrings with real bugs: prose trailing a `Parameters` section (we were rendering parameters called `The`, `you` and `To`), types dropped because numpydoc needs spaces around the colon, `num_partitions, default sqrt(num_rows)` parsing as a list of names and inventing a `default` parameter, and one parameter indented five spaces. `mkdocs build --strict` now exits 0. --- #3747 (the coverage test that keeps this from happening again) is stacked on this branch, so review it after this one. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Will Jones
GitHub
@@ -51,6 +51,11 @@ plugins:
|
||||
paths: [../python/python]
|
||||
options:
|
||||
docstring_style: numpy
|
||||
docstring_options:
|
||||
# Attributes documented in a `Parameters` section, and pydantic
|
||||
# dataclasses whose `__init__` griffe cannot see statically, both
|
||||
# trip this check. It reports nothing actionable here.
|
||||
warn_unknown_params: false
|
||||
heading_level: 3
|
||||
show_signature_annotations: true
|
||||
show_root_heading: true
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Contributing to LanceDB Typescript
|
||||
|
||||
This document outlines the process for contributing to LanceDB Typescript.
|
||||
For general contribution guidelines, see [CONTRIBUTING.md](../CONTRIBUTING.md).
|
||||
For general contribution guidelines, see [CONTRIBUTING.md](https://github.com/lancedb/lancedb/blob/main/CONTRIBUTING.md).
|
||||
|
||||
## Project layout
|
||||
|
||||
|
||||
+114
-49
@@ -26,6 +26,18 @@ is also an [asynchronous API client](#connections-asynchronous).
|
||||
|
||||
::: lancedb.db.DBConnection
|
||||
|
||||
::: lancedb.Session
|
||||
|
||||
## Namespaces (Synchronous)
|
||||
|
||||
A namespace-backed connection resolves tables through a
|
||||
[Lance namespace](https://lancedb.github.io/lance-namespace/) service instead of
|
||||
listing a storage directory.
|
||||
|
||||
::: lancedb.connect_namespace
|
||||
|
||||
::: lancedb.namespace.LanceNamespaceDBConnection
|
||||
|
||||
## Tables (Synchronous)
|
||||
|
||||
::: lancedb.table.Table
|
||||
@@ -34,8 +46,12 @@ is also an [asynchronous API client](#connections-asynchronous).
|
||||
|
||||
::: lancedb.table.FragmentSummaryStats
|
||||
|
||||
::: lancedb.table.TableStatistics
|
||||
|
||||
::: lancedb.table.Tags
|
||||
|
||||
::: lancedb.table.Branches
|
||||
|
||||
## Expressions
|
||||
|
||||
Type-safe expression builder for filters and projections. Use these instead
|
||||
@@ -62,29 +78,46 @@ of raw SQL strings with [where][lancedb.query.LanceQueryBuilder.where] and
|
||||
|
||||
::: lancedb.query.LanceHybridQueryBuilder
|
||||
|
||||
::: lancedb.query.LanceEmptyQueryBuilder
|
||||
|
||||
::: lancedb.query.LanceTakeQueryBuilder
|
||||
|
||||
## Full text queries
|
||||
|
||||
Structured full text queries can be passed to
|
||||
[Table.search][lancedb.table.Table.search] or
|
||||
[AsyncTable.search][lancedb.table.AsyncTable.search] in place of a query string,
|
||||
and combined with [BooleanQuery][lancedb.query.BooleanQuery].
|
||||
|
||||
::: lancedb.query.FullTextQuery
|
||||
|
||||
::: lancedb.query.MatchQuery
|
||||
|
||||
::: lancedb.query.PhraseQuery
|
||||
|
||||
::: lancedb.query.BoostQuery
|
||||
|
||||
::: lancedb.query.MultiMatchQuery
|
||||
|
||||
::: lancedb.query.BooleanQuery
|
||||
|
||||
::: lancedb.query.FullTextOperator
|
||||
|
||||
::: lancedb.query.Occur
|
||||
|
||||
## Embeddings
|
||||
|
||||
::: lancedb.embeddings.registry.EmbeddingFunctionRegistry
|
||||
|
||||
::: lancedb.embeddings.base.EmbeddingFunctionConfig
|
||||
|
||||
::: lancedb.embeddings.base.EmbeddingFunction
|
||||
|
||||
::: lancedb.embeddings.base.TextEmbeddingFunction
|
||||
|
||||
::: lancedb.embeddings.sentence_transformers.SentenceTransformerEmbeddings
|
||||
|
||||
::: lancedb.embeddings.openai.OpenAIEmbeddings
|
||||
|
||||
::: lancedb.embeddings.open_clip.OpenClipEmbeddings
|
||||
::: lancedb.embeddings
|
||||
options:
|
||||
show_root_heading: false
|
||||
show_root_toc_entry: false
|
||||
|
||||
## Remote configuration
|
||||
|
||||
::: lancedb.remote.ClientConfig
|
||||
|
||||
::: lancedb.remote.TimeoutConfig
|
||||
|
||||
::: lancedb.remote.RetryConfig
|
||||
::: lancedb.remote
|
||||
options:
|
||||
show_root_heading: false
|
||||
show_root_toc_entry: false
|
||||
|
||||
## Context
|
||||
|
||||
@@ -122,7 +155,22 @@ tokens = list(lancedb.tokenize("acme makes searchable data",
|
||||
custom_stop_words=["acme"]))
|
||||
```
|
||||
|
||||
::: lancedb.index.FTS
|
||||
::: lancedb.tokenize
|
||||
|
||||
::: lancedb.FtsToken
|
||||
|
||||
## Blobs
|
||||
|
||||
Blob columns store large binary values out of line so they can be read lazily
|
||||
instead of being materialized with the rest of the row.
|
||||
|
||||
::: lancedb.blob
|
||||
|
||||
::: lancedb.BlobType
|
||||
|
||||
::: lancedb._blob.BlobFile
|
||||
options:
|
||||
show_root_full_path: false
|
||||
|
||||
## Utilities
|
||||
|
||||
@@ -130,6 +178,14 @@ tokens = list(lancedb.tokenize("acme makes searchable data",
|
||||
|
||||
::: lancedb.merge.LanceMergeInsertBuilder
|
||||
|
||||
::: lancedb.otel.instrument_lancedb_metrics
|
||||
|
||||
## Exceptions
|
||||
|
||||
::: lancedb.exceptions.MissingValueError
|
||||
|
||||
::: lancedb.exceptions.MissingColumnError
|
||||
|
||||
## Integrations
|
||||
|
||||
## Pydantic
|
||||
@@ -138,19 +194,30 @@ tokens = list(lancedb.tokenize("acme makes searchable data",
|
||||
|
||||
::: lancedb.pydantic.vector
|
||||
|
||||
::: lancedb.pydantic.Vector
|
||||
|
||||
::: lancedb.pydantic.MultiVector
|
||||
|
||||
::: lancedb.pydantic.LanceModel
|
||||
|
||||
## PyTorch
|
||||
|
||||
::: lancedb.streaming.StreamingDataset
|
||||
|
||||
::: lancedb.permutation.permutation_builder
|
||||
|
||||
::: lancedb.permutation.PermutationBuilder
|
||||
|
||||
::: lancedb.permutation.Permutation
|
||||
|
||||
::: lancedb.permutation.Transforms
|
||||
|
||||
## Reranking
|
||||
|
||||
::: lancedb.rerankers.linear_combination.LinearCombinationReranker
|
||||
|
||||
::: lancedb.rerankers.cohere.CohereReranker
|
||||
|
||||
::: lancedb.rerankers.colbert.ColbertReranker
|
||||
|
||||
::: lancedb.rerankers.cross_encoder.CrossEncoderReranker
|
||||
|
||||
::: lancedb.rerankers.openai.OpenaiReranker
|
||||
::: lancedb.rerankers
|
||||
options:
|
||||
show_root_heading: false
|
||||
show_root_toc_entry: false
|
||||
|
||||
## Connections (Asynchronous)
|
||||
|
||||
@@ -161,6 +228,12 @@ can be used to create, list, or open tables.
|
||||
|
||||
::: lancedb.db.AsyncConnection
|
||||
|
||||
## Namespaces (Asynchronous)
|
||||
|
||||
::: lancedb.connect_namespace_async
|
||||
|
||||
::: lancedb.namespace.AsyncLanceNamespaceDBConnection
|
||||
|
||||
## Tables (Asynchronous)
|
||||
|
||||
Table hold your actual data as a collection of records / rows.
|
||||
@@ -169,32 +242,20 @@ Table hold your actual data as a collection of records / rows.
|
||||
|
||||
::: lancedb.table.AsyncTags
|
||||
|
||||
::: lancedb.table.AsyncBranches
|
||||
|
||||
## Indices (Asynchronous)
|
||||
|
||||
Indices can be created on a table to speed up queries. This section
|
||||
lists the indices that LanceDb supports.
|
||||
|
||||
::: lancedb.index.BTree
|
||||
|
||||
::: lancedb.index.Bitmap
|
||||
|
||||
::: lancedb.index.LabelList
|
||||
|
||||
::: lancedb.index.FTS
|
||||
|
||||
::: lancedb.index.IvfPq
|
||||
|
||||
::: lancedb.index.HnswPq
|
||||
|
||||
::: lancedb.index.HnswSq
|
||||
|
||||
::: lancedb.index.IvfFlat
|
||||
|
||||
::: lancedb.index.IvfSq
|
||||
|
||||
::: lancedb.index.IvfRq
|
||||
|
||||
::: lancedb.index.HnswFlat
|
||||
::: lancedb.index
|
||||
options:
|
||||
show_root_heading: false
|
||||
show_root_toc_entry: false
|
||||
# `lang_mapping` is defined in the module rather than imported, so it is
|
||||
# picked up despite not being in `__all__`. It is an internal lookup table.
|
||||
filters: ["!^_", "!^lang_mapping$"]
|
||||
|
||||
::: lancedb.table.IndexStatistics
|
||||
|
||||
@@ -222,3 +283,7 @@ rows nearest to a query vector and can be created with the
|
||||
::: lancedb.query.AsyncHybridQuery
|
||||
options:
|
||||
inherited_members: true
|
||||
|
||||
::: lancedb.query.AsyncTakeQuery
|
||||
options:
|
||||
inherited_members: true
|
||||
|
||||
Reference in New Issue
Block a user