mirror of https://github.com/lancedb/lancedb.git synced 2026-08-18 20:18:37 +00:00

Files

T

Will Jones 5a1015ba72 docs(python): fill gaps in the Python API reference (#3746 )

`docs/src/python/python.md` is the whole Python API reference, but it is
maintained by hand and had drifted from the public API. Anything not
listed there simply doesn't get rendered, so a number of public,
documented, tested APIs were invisible to users — most notably branch
management, where `diff` and `merge` live.

I audited every public symbol reachable from `lancedb` and its
subpackages against the `:::` directives on the page. This adds the
missing ones:

- **Branching** — `Branches`, `AsyncBranches` (`list` / `create` /
`checkout` / `delete` / `diff` / `merge`)
- **Tables** — `TableStatistics` (returned by `Table.stats()`; the
fragment-level stats classes were already listed)
- **Full text queries** — `FullTextQuery`, `MatchQuery`, `PhraseQuery`,
`BoostQuery`, `MultiMatchQuery`, `BooleanQuery`, `FullTextOperator`,
`Occur`
- **Querying** — `LanceEmptyQueryBuilder`, `LanceTakeQueryBuilder`,
`AsyncTakeQuery`
- **Indices** — `Fm` (the FM-index for substring search), `IndexConfig`
- **Blobs** — `blob`, `BlobType`, `BlobFile`
- **Namespaces** — `connect_namespace`, `connect_namespace_async`, and
both namespace connection classes
- **Remote config** — `TlsConfig`, `HeaderProvider`, `OAuthConfig`,
`OAuthFlowType`
- **Rerankers** — the `Reranker` base class plus `JinaReranker`,
`RRFReranker`, `MRRReranker`, `AnswerdotaiRerankers`,
`VoyageAIReranker`, `WatsonxReranker` (5 of 12 were listed)
- **Embeddings** — `get_registry`, `register`, and the 14 embedding
functions that were missing (3 of 17 were listed)
- **PyTorch** — `StreamingDataset` and the permutation API it is built
on
- **Misc** — `Session`, `tokenize`, `FtsToken`, `pydantic.Vector`,
`pydantic.MultiVector`, `instrument_lancedb_metrics`, and the two
exception types

It also repairs cross-references in docstrings that no longer resolve:
links into guide pages that have since moved to lancedb.com
(`querying-an-ann-index`, `experimental-full-text-search`),
`lance.dataset` references with no inventory behind them, and the
relative targets `[Table](Table)` and `[PyArrow Table](pyarrow.Table)`.

Deliberately left out: concrete implementation classes reached through
their abstract base (`LanceTable`, `LanceDBConnection`,
`RemoteDBConnection`), query base classes already covered by
`inherited_members: true`, and internal plumbing such as
`FullTextSearchQuery` and `ColumnOrdering`.

## Testing

The docs job only runs on pushes to `main`, so I built the site locally
and compared against a build of `upstream/main`: every added entry
resolves, and no symbol that was rendered before stopped being rendered
when the four packages moved to automodule. `mkdocs build --strict`
exits 0 on this branch, against 61 warnings on `main`.

## Also in this PR

`lancedb.index`, `lancedb.embeddings`, `lancedb.remote` and
`lancedb.rerankers` are now rendered by a single mkdocstrings directive
each, driven by the module's `__all__`, rather than a hand-maintained
list. These four are where most of the drift was, and `__all__` is
harder to forget than a docs page. `lancedb.embeddings` had no
`__all__`; without one mkdocstrings renders no members at all for a
re-export package, so one is added. AGENTS.md gains a section on how the
page is wired up and how to build the docs locally.

Rendering all that code for the first time surfaced ~100 more build
warnings, which would have made #3707 (turning on `mkdocs build
--strict`) harder to land, so the warning backlog is cleared here too.
97 of the 158 warnings were one systematic false positive — griffe
cannot see the generated `__init__` of a pydantic dataclass, so every
documented parameter looks unknown — switched off via
`warn_unknown_params`. The remaining 61 came from 15 docstrings with
real bugs: prose trailing a `Parameters` section (we were rendering
parameters called `The`, `you` and `To`), types dropped because numpydoc
needs spaces around the colon, `num_partitions, default sqrt(num_rows)`
parsing as a list of names and inventing a `default` parameter, and one
parameter indented five spaces. `mkdocs build --strict` now exits 0.

---

#3747 (the coverage test that keeps this from happening again) is
stacked on this branch, so review it after this one.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-07-30 16:50:05 -07:00

1.8 KiB

Raw Permalink Blame History

Contributing to LanceDB Typescript

This document outlines the process for contributing to LanceDB Typescript. For general contribution guidelines, see CONTRIBUTING.md.

Project layout

The Typescript package is a wrapper around the Rust library, lancedb. We use the napi-rs library to create the bindings between Rust and Typescript.

src/: Rust bindings source code
lancedb/: Typescript package source code
__test__/: Unit tests
examples/: A pnpm package with the examples shown in the documentation

Development environment

To set up your development environment, you will need to install the following:

Node.js 22 or later (required by pnpm 11)
pnpm 11 or later (or run via corepack enable, which uses the packageManager field in package.json)
Rust's package manager, Cargo. Use rustup to install.
protoc (Protocol Buffers compiler)

Initial setup:

pnpm install

Commit Hooks

It is highly recommended to install the pre-commit hooks to ensure that your code is formatted correctly and passes basic checks before committing:

pre-commit install

Development

Most common development commands can be run using the pnpm scripts.

Build the package

pnpm install
pnpm build

Lint:

pnpm lint

Format and fix lints:

pnpm lint-fix

Run tests:

pnpm test

To run a single test:

# Single file: table.test.ts
pnpm test -- table.test.ts
# Single test: 'merge insert' in table.test.ts
pnpm test -- table.test.ts --testNamePattern=merge\ insert

1.8 KiB Raw Permalink Blame History