A broad documentation pass across the areas LanceDB Enterprise and Embedded diverge on most: **Data Curation** -- GPU Function execution, Slurm scheduling (and what Slurm is, for readers new to it), blob late materialization as its own page, joins, aggregation/profiling, sampling, near-duplicate detection, chunking, distributed shuffle, and IVF-partition clustering. **Feature Engineering** -- computed columns and materialized views, restructured to lead with the Function-based path and show the SQL form side by side with Python throughout; a single Auto Refresh page instead of duplicating scheduling details on every refresh page; adaptive execution; checkpointing; GPU and Slurm placement for refresh workers; a Training Cache page for the PyTorch data loader's automatic cache benefit. **Security & Governance** -- an Audit Log reference (schema, access, grants), Secrets promoted to an ordinary catalog object with their own SQL DDL and a vending/rotation guide, and privilege-table fixes verified against the authorization source. **Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry moved into a real Deployment area; Automated Cleanup folded into Intelligent Auto-Maintenance (every other maintenance decision already lived there); RTE Compaction and Advanced Conflict Resolution documented in depth. **Landing page** -- the Enterprise feature comparison table reorganized (several redundant or stale rows removed, several new areas added), and the "why Enterprise" bullets replaced with ones tied to a concrete page (Performance, Scale, Fault tolerance, Multicloud deployment, Simplify operations) instead of vague promises. Plus a full pass addressing lancedb-gatekeeper's review: a real test-safety bug (a quickstart test could silently overwrite and delete a deployment's own `frames` table), a fabricated SQL statement with no backing implementation, an actual dedup-guarantee bug in `take_row_ids`, and dozens of documentation-accuracy fixes verified against source rather than taken on faith -- each either fixed, or left with an explicit note on why it needs a feature or deployment owner's sign-off instead of being guessed at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
8.2 KiB
Contributing to LanceDB
LanceDB is an open-source project and we welcome contributions from the community. This document outlines the process for contributing to LanceDB.
Reporting Issues
If you encounter a bug or have a feature request, please open an issue on the GitHub issue tracker.
Picking an issue
We track issues on the GitHub issue tracker. If you are looking for something to work on, check the good first issue label. These issues are typically the best described and have the smallest scope.
If there's an issue you are interested in working on, please leave a comment on the issue. This will help us avoid duplicate work. Additionally, if you have questions about the issue, please ask them in the issue comments. We are happy to provide guidance on how to approach the issue.
Configuring Git
First, fork the repository on GitHub, then clone your fork:
git clone https://github.com/<username>/lancedb.git
cd lancedb
Then add the main repository as a remote:
git remote add upstream https://github.com/lancedb/lancedb.git
git fetch upstream
Setting up your development environment
We have development environments for Python, Typescript, and Java. Each environment has its own setup instructions.
Best practices for pull requests
For the best chance of having your pull request accepted, please follow these guidelines:
- Unit test all bug fixes and new features. Your code will not be merged if it doesn't have tests.
- If you change the public API, update the documentation in the
docsdirectory. - Aim to minimize the number of changes in each pull request. Keep to solving one problem at a time, when possible.
- Before marking a pull request ready-for-review, do a self review of your code. Is it clear why you are making the changes? Are the changes easy to understand?
- Use conventional commit messages as pull request titles. Examples:
- New feature:
feat: adding foo API - Bug fix:
fix: issue with foo API - Documentation change:
docs: adding foo API documentation
- New feature:
- If your pull request is a work in progress, leave the pull request as a draft. We will assume the pull request is ready for review when it is opened.
- When writing tests, test the error cases. Make sure they have understandable error messages.
Project structure
The core library is written in Rust. The Python, Typescript, and Java libraries are wrappers around the Rust library.
src/lancedb: Rust library source codepython: Python package source codenodejs: Typescript package source codenode: Deprecated Typescript package source codejava: Java package source codedocs/src: SDK reference, built with mkdocsdocs/web: website guides for LanceDB Embedded and the APIs used by both Embedded and Enterprisedocs/web-tests: the tests the documentation's code examples are extracted from
Documentation
Update the docs in the same pull request as the behavior they describe. You can merge docs before a feature launches; keep the unfinished material hidden until it is ready for readers.
Choose the right place
docs/web/holds website guides for LanceDB Embedded and the APIs used by both Embedded and Enterprise.docs/src/holds the SDK reference. See docs/README.md for its build and API-generation instructions.docs/web-tests/andrust/lancedb/tests/hold tests used to generate the website's code examples.
Document SDK APIs here, and explain which operations work in Embedded and Enterprise. Put Enterprise-specific deployment and operating instructions in the Enterprise documentation.
Some pages are placeholders that the website build replaces with an Enterprise page. Changes to a placeholder won't appear on the combined site. Put instructions that both products need on a separate page and link to it.
Preview and check your change
From the repository root:
npm i -g mint@4.2.888
(cd docs/web && mint dev)
This previews this checkout, including your uncommitted changes. It doesn't include the other documentation sources or render the availability labels. Use the combined preview when checking Enterprise replacements or links to pages from another source. Staging shows saved release editions, so it won't show a new source change just because that change merged.
Follow the style guide. Vale checks wording, names and punctuation using the bundled LanceDB rules. Pull requests show suggestions on the lines they change. With Vale 3.24.0 installed, check a page locally with:
vale docs/web/indexing/vector-index.mdx
Keep existing {#anchor} values when changing headings, so links to those
sections keep working. Give new headings stable anchors. For links in rendered
components, write the path directly, such as href="/quickstart", and spell out
attributes rather than using property spreads. That lets versioned docs keep
links inside the selected edition. Code examples aren't subject to this rule.
Code examples
Edit the test that produces an example, then regenerate its snippet. The
sources are docs/web-tests/{py,ts,rs}, docs/web-tests/blob-v2/{py,ts,rs} and
rust/lancedb/tests. Run this from the repository root:
uv run docs/web-tests/mdx_snippets_gen.py \
-s docs/web-tests/py -s docs/web-tests/ts -s docs/web-tests/rs \
-s docs/web-tests/blob-v2/py -s docs/web-tests/blob-v2/ts -s docs/web-tests/blob-v2/rs \
-s rust/lancedb/tests -o docs/web/snippets
Run the tests for the examples you changed; generating a snippet doesn't run its example. Review the generated diff and commit only the relevant snippets with their tests. Don't assume the PR checks execute every documentation example.
Say where a feature works
Use availability in a page's frontmatter to describe the support covered by
that page. Each product (embedded for LanceDB Embedded, enterprise for
LanceDB Enterprise) is available or unavailable. Use the body for setup
requirements, SDK differences and unsupported operations. For example, a page
about signing in to Enterprise could declare:
availability:
embedded: unavailable
enterprise: available
Check these claims with the feature owner. Don't use varies, treat an API
stub as working support, or mark a supported feature unavailable just because
it needs configuration. If one label can't describe the whole page accurately,
make the page's scope narrower and explain the differences.
Assembly turns this declaration into a page label and, for a feature supported by only one product, a sidebar tag. Don't add those badges by hand. Availability labels do not hide unreleased content.
Write docs before launch
You don't need to hold a documentation PR until launch. Use these controls:
- For a whole page, add its path to
docs/web/.mintignore. Leave it out ofdocs/web/docs.json, redirects and links from published pages. Removing a sidebar entry alone doesn't hide a page. - For a section, wrap it in an MDX comment,
{/* ... */}, with a note such asDraft for the Feature X release.Put draft table rows and list items in a comment below the table or list, not between entries.
Drafts remain in the source. Use them to avoid advertising an unfinished feature, not to keep information confidential. See Writing docs before release for examples and how to preview a draft.
Before the intended release, remove the draft controls, restore the navigation
and links, and check the result. Those changes must be in the source commit the
edition uses. A later change to main doesn't update an edition already saved.
For a mistake in an existing edition, fix the source and request a
recorded correction.
Release process
For information on the release process, see: release_process.md