mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-14 15:52:26 +00:00
Jack YeandBrendan Clement
47b7babc08
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).
This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.
Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.
**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.
**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.
Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.
**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.
The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.
---------
Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
74 lines
3.0 KiB
Plaintext
74 lines
3.0 KiB
Plaintext
---
|
|
title: "GenKit"
|
|
sidebarTitle: "GenKit"
|
|
|
|
---
|
|
|
|
import {
|
|
TsFrameworksGenkitCustomIndexer,
|
|
TsFrameworksGenkitCustomRetriever,
|
|
TsFrameworksGenkitUsage,
|
|
} from '/snippets/integrations.mdx';
|
|
|
|
### genkitx-lancedb {#genkitx-lancedb}
|
|
Genkit is an open-source framework for building end-to-end AI and RAG pipelines with a clean, TypeScript-first
|
|
developer experience. The genkitx-lancedb plugin lets you use LanceDB as a high-performance vector store
|
|
inside your Genkit flows, so you can index, search, and retrieve data efficiently as part of your AI
|
|
applications.
|
|
|
|
### Installation {#installation}
|
|
```bash
|
|
pnpm install genkitx-lancedb
|
|
```
|
|
|
|
### Usage {#usage}
|
|
|
|
Adding LanceDB plugin to your genkit instance.
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsFrameworksGenkitUsage}
|
|
</CodeBlock>
|
|
|
|
You can run this app with the following command:
|
|
```bash
|
|
genkit start -- tsx --watch src/index.ts
|
|
```
|
|
|
|
This'll add LanceDB as a retriever and indexer to the genkit instance. You can see it in the GUI view
|
|
<img width="1710" alt="Screenshot 2025-05-11 at 7 21 05 PM" src="https://github.com/user-attachments/assets/e752f7f4-785b-4797-a11e-72ab06a531b7" />
|
|
|
|
**Testing retrieval on a sample table**
|
|
Let's see the raw retrieval results
|
|
|
|
<img width="1710" alt="Screenshot 2025-05-11 at 7 21 05 PM" src="https://github.com/user-attachments/assets/b8d356ed-8421-4790-8fc0-d6af563b9657" />
|
|
On running this query, you'll get 5 results fetched from the lancedb table, where each result looks something like this:
|
|
<img width="1417" alt="Screenshot 2025-05-11 at 7 21 18 PM" src="https://github.com/user-attachments/assets/77429525-36e2-4da6-a694-e58c1cf9eb83" />
|
|
|
|
|
|
|
|
## Creating a custom RAG flow {#creating-a-custom-rag-flow}
|
|
|
|
Now that we've seen how you can use LanceDB in a Genkit pipeline, let's refine the flow and create a RAG. A RAG flow will consist of an index and a retriever with its outputs postprocessed and fed into an LLM for final response
|
|
|
|
### Creating custom indexer flows {#creating-custom-indexer-flows}
|
|
You can also create custom indexer flows, utilizing more options and features provided by LanceDB.
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsFrameworksGenkitCustomIndexer}
|
|
</CodeBlock>
|
|
|
|
<img width="1316" alt="Screenshot 2025-05-11 at 8 35 56 PM" src="https://github.com/user-attachments/assets/e2a20ce4-d1d0-4fa2-9a84-f2cc26e3a29f" />
|
|
|
|
In your console, you can see the logs
|
|
|
|
<img width="511" alt="Screenshot 2025-05-11 at 7 19 14 PM" src="https://github.com/user-attachments/assets/243f26c5-ed38-40b6-b661-002f40f0423a" />
|
|
|
|
### Creating custom retriever flows {#creating-custom-retriever-flows}
|
|
You can also create custom retriever flows, utilizing more options and features provided by LanceDB.
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsFrameworksGenkitCustomRetriever}
|
|
</CodeBlock>
|
|
Now using our retrieval flow, we can ask a question about the ingested PDF
|
|
<img width="1306" alt="Screenshot 2025-05-11 at 7 18 45 PM" src="https://github.com/user-attachments/assets/86c66b13-7c12-4d5f-9d81-ae36bfb1c346" />
|
|
|