b731a6aed9
- Creates testing files `md_testing.py` and `md_testing.js` for testing python and nodejs code in markdown files in the documentation This listens for HTML tags as well: `<!--[language] code code code...-->` will create a set-up file to create some mock tables or to fulfill some assumptions in the documentation. - Creates a github action workflow that triggers every push/pr to `docs/**` - Modifies documentation so tests run (mostly indentation, some small syntax errors and some missing imports) A list of excluded files that we need to take a closer look at later on: ```javascript const excludedFiles = [ "../src/fts.md", "../src/embedding.md", "../src/examples/serverless_lancedb_with_s3_and_lambda.md", "../src/examples/serverless_qa_bot_with_modal_and_langchain.md", "../src/examples/youtube_transcript_bot_with_nodejs.md", ]; ``` Many of them can't be done because we need the OpenAI API key :(. `fts.md` has some issues with the library, I believe this is still experimental? Closes #170 --------- Co-authored-by: Will Jones <willjones127@gmail.com>
3.2 KiB
Welcome to LanceDB's Documentation
LanceDB is an open-source database for vector-search built with persistent storage, which greatly simplifies retrevial, filtering and management of embeddings.
The key features of LanceDB include:
-
Production-scale vector search with no servers to manage.
-
Store, query and filter vectors, metadata and multi-modal data (text, images, videos, point clouds, and more).
-
Support for vector similarity search, full-text search and SQL.
-
Native Python and Javascript/Typescript support.
-
Zero-copy, automatic versioning, manage versions of your data without needing extra infrastructure.
-
Ecosystem integrations with LangChain 🦜️🔗, LlamaIndex 🦙, Apache-Arrow, Pandas, Polars, DuckDB and more on the way.
LanceDB's core is written in Rust 🦀 and is built using Lance, an open-source columnar format designed for performant ML workloads.
Quick Start
=== "Python"
shell pip install lancedb
```python
import lancedb
uri = "data/sample-lancedb"
db = lancedb.connect(uri)
table = db.create_table("my_table",
data=[{"vector": [3.1, 4.1], "item": "foo", "price": 10.0},
{"vector": [5.9, 26.5], "item": "bar", "price": 20.0}])
result = table.search([100, 100]).limit(2).to_df()
```
=== "Javascript"
shell npm install vectordb
```javascript
const lancedb = require("vectordb");
const uri = "data/sample-lancedb";
const db = await lancedb.connect(uri);
const table = await db.createTable("my_table",
[{ id: 1, vector: [3.1, 4.1], item: "foo", price: 10.0 },
{ id: 2, vector: [5.9, 26.5], item: "bar", price: 20.0 }])
const results = await table.search([100, 100]).limit(2).execute();
```
Complete Demos (Python)
- YouTube Transcript Search
- Documentation QA Bot using LangChain
- Multimodal search using CLIP
- Serverless QA Bot with S3 and Lambda
- Serverless QA Bot with Modal
Complete Demos (JavaScript)
Documentation Quick Links
Basic Operations- basic functionality of LanceDB.Embedding Functions- functions for working with embeddings.Indexing- create vector indexes to speed up queries.Full text search- [EXPERIMENTAL] full-text search APIEcosystem Integrations- integrating LanceDB with python data tooling ecosystem.Python API Reference- detailed documentation for the LanceDB Python SDK.Node API Reference- detailed documentation for the LanceDB Python SDK.