mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-20 17:43:58 +00:00
## Summary Fixes the problems a page-by-page review of the docs found before launch, after checking each one against the current source and, where noted, the code. The matching fixes for the dataset and Geneva pages, which live in `lancedb/docs`, are in lancedb/docs#402. **Tables** - `integrations/embedding/cohere`: each input-type description was split across two table rows and wrapped in literal quotes (`"`search_document`"`). Now one row per type. - `integrations/embedding/gemini`: removes the same stray quotes from the task-type table. - `integrations/ai/langchain`: the MMR search table's `kwargs` row was missing a cell, so its description sat under Type. - Six reranker pages (answerdotai, cohere, colbert, jina, openai, voyageai): closes the backtick before `_relevance_score`. - `indexing/vector-index`: finishes a cell cut off at "with scalar". - `sql/dql/dedupe`: labels the empty header cell in the resolution table. **Corrections** - `curation/sampling`: sampling works in LanceDB Embedded through the Python SDK, so the page is available in both, with the SQL sections marked Enterprise. - `feature-engineering/index`, `add-computed-column`: automatic refresh is on by default for each computed column and takes effect once a deployment enables its scheduler. `refresh-column` already said so; these two implied it was off. - `indexing/scalar-index`, `indexing/index`: `LABEL_LIST` accelerates `array_has_all`/`array_has_any`; `array_contains_*` doesn't exist. Also merges scalar-index's two "nested fields" sections into one. - `integrations/embedding/ibm`: lists the four current models from `CURRENT_MODELS` (it listed a deprecated one) and explains that the default stays a legacy ID so existing tables reload with their model. - `integrations/reranking/openai`: the class is `OpenaiReranker`, not `OpenAI()`. - `integrations/ai/synthetic-data-kit`: the CLI is `synthetic-data-kit` throughout, each step runs one command, each step passes the file it writes to the next, and `save-as` uses `-f`/`-o`. - `integrations/data/dlt`: `secrets.toml` defines `api_token`, which the pipeline reads, and the run command runs the file the page creates. - `search/vector-search`: the multivector walkthrough connected to Enterprise in step 1 and a local path in step 2. It now runs locally throughout, with a note on connecting to Enterprise, and is renamed so it doesn't read as a duplicate of the short section above. **Structure and wording** - `integrations/embedding/nvidia`: adds headings, and drops test-suite details aimed at contributors. - `integrations/ai/genkit`: makes the retrieval test a real heading and describes the query step. - `integrations/data/voxel51`: replaces a raw MkDocs `!!! note` with a `<Note>`. - `sql/ddl/table`, `sql/ddl/job`: "Errors and limits" is a real heading, not bold text. - `curation/windowing`: sidebar title matches the curation overview ("Rank and Deduplicate"). - `deployment/storage`, `basics/agentic-skills`: fixes a garbled sentence and a stale reference to the quickstart. **Checked:** no new Vale alerts on the changed pages. Assembled together with lancedb/docs#402, the site passes `mint validate` and `mint broken-links --check-anchors --check-redirects`. Every changed page was reviewed in a local preview. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
78 lines
3.1 KiB
Plaintext
78 lines
3.1 KiB
Plaintext
---
|
|
title: "GenKit"
|
|
sidebarTitle: "GenKit"
|
|
|
|
---
|
|
|
|
import {
|
|
TsFrameworksGenkitCustomIndexer,
|
|
TsFrameworksGenkitCustomRetriever,
|
|
TsFrameworksGenkitUsage,
|
|
} from '/snippets/integrations.mdx';
|
|
|
|
### genkitx-lancedb {#genkitx-lancedb}
|
|
Genkit is an open-source framework for building end-to-end AI and RAG pipelines with a clean, TypeScript-first
|
|
developer experience. The genkitx-lancedb plugin lets you use LanceDB as a high-performance vector store
|
|
inside your Genkit flows, so you can index, search, and retrieve data efficiently as part of your AI
|
|
applications.
|
|
|
|
### Installation {#installation}
|
|
```bash
|
|
pnpm install genkitx-lancedb
|
|
```
|
|
|
|
### Usage {#usage}
|
|
|
|
Adding LanceDB plugin to your genkit instance.
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsFrameworksGenkitUsage}
|
|
</CodeBlock>
|
|
|
|
You can run this app with the following command:
|
|
```bash
|
|
genkit start -- tsx --watch src/index.ts
|
|
```
|
|
|
|
This'll add LanceDB as a retriever and indexer to the genkit instance. You can see it in the GUI view
|
|
<img width="1710" alt="Screenshot 2025-05-11 at 7 21 05 PM" src="https://github.com/user-attachments/assets/e752f7f4-785b-4797-a11e-72ab06a531b7" />
|
|
|
|
### Testing retrieval on a sample table {#testing-retrieval-on-a-sample-table}
|
|
|
|
To see the raw retrieval results, open the LanceDB retriever in the Genkit Developer UI,
|
|
enter a query and run it:
|
|
|
|
<img width="1710" alt="Screenshot 2025-05-11 at 7 21 05 PM" src="https://github.com/user-attachments/assets/b8d356ed-8421-4790-8fc0-d6af563b9657" />
|
|
|
|
The retriever returns 5 results from the LanceDB table. Each result looks something like this:
|
|
|
|
<img width="1417" alt="Screenshot 2025-05-11 at 7 21 18 PM" src="https://github.com/user-attachments/assets/77429525-36e2-4da6-a694-e58c1cf9eb83" />
|
|
|
|
|
|
|
|
## Creating a custom RAG flow {#creating-a-custom-rag-flow}
|
|
|
|
Now that we've seen how you can use LanceDB in a Genkit pipeline, let's refine the flow and create a RAG. A RAG flow will consist of an index and a retriever with its outputs postprocessed and fed into an LLM for final response
|
|
|
|
### Creating custom indexer flows {#creating-custom-indexer-flows}
|
|
You can also create custom indexer flows, using more options and features provided by LanceDB.
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsFrameworksGenkitCustomIndexer}
|
|
</CodeBlock>
|
|
|
|
<img width="1316" alt="Screenshot 2025-05-11 at 8 35 56 PM" src="https://github.com/user-attachments/assets/e2a20ce4-d1d0-4fa2-9a84-f2cc26e3a29f" />
|
|
|
|
In your console, you can see the logs
|
|
|
|
<img width="511" alt="Screenshot 2025-05-11 at 7 19 14 PM" src="https://github.com/user-attachments/assets/243f26c5-ed38-40b6-b661-002f40f0423a" />
|
|
|
|
### Creating custom retriever flows {#creating-custom-retriever-flows}
|
|
You can also create custom retriever flows, using more options and features provided by LanceDB.
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsFrameworksGenkitCustomRetriever}
|
|
</CodeBlock>
|
|
Now using our retrieval flow, we can ask a question about the ingested PDF
|
|
<img width="1306" alt="Screenshot 2025-05-11 at 7 18 45 PM" src="https://github.com/user-attachments/assets/86c66b13-7c12-4d5f-9d81-ae36bfb1c346" />
|
|
|