Jack Ye
5b5ed930ac
Every group now answers exactly one question. Get started and Concepts cover what the system is; the verb groups cover doing something with it; Operate, Secure and Deploy cover running it; SQL reference is lookup. Maintenance becomes Operate, Governance becomes Secure, and the rest take verbs to match. Three new pages in Concepts and Get started: Guarantees states what a write promises when it returns, which version a reader sees, what survives a failure, and -- the half usually missing -- what is not guaranteed: no multi-table transactions, no read-your-writes across processes without an interval, unenforced primary keys that do not enforce, no background compaction. Limits collects bounds that were stated in passing across a dozen pages, with the numbers verified against the code rather than inferred: name lengths, job event caps, SHOW ERRORS bounds, the sampling range that is refused rather than clamped. Architecture illustrates embedded against distributed as the one deployment decision, and is written to cover both rather than only Enterprise. Indexing splits by question. Choosing an index is a Query decision and moves there; building, repairing and GPU-building it stay in Operate. SQL becomes a top-level reference of fifteen pages instead of a sub-group of Query. Auditing sophon-sql turned up a large undocumented surface: ALTER TABLE ADD COLUMN in its plain, computed and function forms with FILTER, IGNORE NULLS and OVER modifiers; SET UNENFORCED PRIMARY KEY; SET and UNSET WAL with sharding; CREATE FUNCTION with its dependency options; REFRESH COLUMN and REFRESH MATERIALIZED VIEW; and the SHOW family. None of it was written down.
LanceDB Documentation
LanceDB docs are available at docs.lancedb.com.
The SDK docs are built and deployed automatically by Github Actions
whenever a commit is pushed to the main branch. So it is possible for the docs to show
unreleased features.
Building the docs
Setup
- Install LanceDB Python. See setup in Python contributing guide.
Run
make developto install the Python package. - Install documentation dependencies. From LanceDB repo root:
pip install -r docs/requirements.txt
Preview the docs
cd docs
mkdocs serve
If you want to just generate the HTML files:
PYTHONPATH=. mkdocs build -f docs/mkdocs.yml
If successful, you should see a docs/site directory that you can verify locally.
Adding examples
To make sure examples are correct, we put examples in test files so they can be run as part of our test suites.
You can see the tests are at:
- Python:
python/python/tests/docs - Typescript:
nodejs/examples/
Checking python examples
cd python
pytest -vv python/tests/docs
Checking typescript examples
The examples depend on @lancedb/lancedb at file:../dist, so the package must be
built before running the tests. This uses pnpm; see the
Typescript contributing guide for the toolchain setup.
pushd nodejs
pnpm install
pnpm build
popd
Then you can run the examples by going to the nodejs/examples directory, which is a
separate pnpm package with its own lockfile:
pushd nodejs/examples
pnpm install
pnpm test
popd
API documentation
Python
The Python API documentation is organized based on the file docs/src/python/python.md.
We manually add entries there so we can control the organization of the reference page.
However, this means any new types must be manually added to the file. No additional
steps are needed to generate the API documentation.
Typescript
The typescript API documentation is generated from the typescript source code using typedoc.
When new APIs are added, you must manually re-run the typedoc command to update the API documentation. The new files should be checked into the repository.
pushd nodejs
# `pnpm docs` would invoke pnpm's built-in `docs` command, not the script.
pnpm run docs
popd