mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-18 01:32:27 +00:00
docs: bring the open-source documentation into this repository
The pages describing this code lived in lancedb/docs, a repository with ten stars that neither contributing guide mentioned. A user who hit a wrong page while reading the source had to find a separate, near-invisible repository to fix it. These 102 pages now sit beside the code, and a documentation fix is a pull request in the repository the reader already had open. `docs/web` is a complete Mintlify site: `cd docs/web && mint dev` serves it with no other checkout. Enterprise pages stay private and are merged in when the published site is assembled, so links into them do not resolve here — every open-source page does. `docs/web-tests` holds the tests the code examples are extracted from. They ran against a published wheel before; here they can run against this repository's own SDK, so an example that stops compiling fails the pull request that broke it rather than being noticed later. Both directories sit beside `docs/src` and are invisible to it: mkdocs reads `docs_dir: src`, so the SDK reference builds and deploys exactly as before.
This commit is contained in:
@@ -0,0 +1,17 @@
|
||||
---
|
||||
title: "RAG and Agents"
|
||||
sidebarTitle: "Notebooks"
|
||||
description: "Explore a variety of RAG (Retrieval-Augmented Generation) and agent applications built with LanceDB."
|
||||
---
|
||||
|
||||
The table below lists example notebooks we've prepared for a variety of RAG (Retrieval-Augmented Generation)
|
||||
and agent applications built with LanceDB.
|
||||
|
||||
| Project | Description |
|
||||
|:----------|:------------|
|
||||
| **Contextual RAG** <br /> <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/Contextual-RAG/Anthropic_Contextual_RAG.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab" /></a>[View on GitHub](https://github.com/lancedb/vectordb-recipes/tree/main/examples/Contextual-RAG) | Improves retrieval by combatting the "lost in the middle" problem. This technique uses an LLM to generate succinct context for each document chunk, then prepends that context to the chunk before embedding, leading to more accurate retrieval. |
|
||||
| **NVIDIA RAG Blueprint with LanceDB** <br /> [Read the tutorial](/tutorials/agents/nvidia-rag-blueprint/)<br />[View on GitHub](https://github.com/lancedb/vectordb-recipes/tree/main/examples/nvidia-rag-blueprint-lancedb) | Shows how to use LanceDB as the retrieval layer for NVIDIA RAG Blueprint with a Docker-first, retrieval-only integration path that includes hybrid search and pluggable rerankers. |
|
||||
| **Matryoshka Embeddings** <br /> <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/RAG-with_MatryoshkaEmbed-Llamaindex/main.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab" /></a>[View on GitHub](https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/RAG-with_MatryoshkaEmbed-Llamaindex) | Demonstrates a RAG pipeline using Matryoshka Embeddings with LanceDB and LlamaIndex. This method allows for efficient storage and retrieval of nested, variable-sized embeddings. |
|
||||
| **HyDE (Hypothetical Document Embeddings)** <br /> <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/Advance-RAG-with-HyDE/main.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab" /></a>[View on GitHub](https://github.com/lancedb/vectordb-recipes/tree/main/examples/Advance-RAG-with-HyDE) | An advanced RAG technique that uses an LLM to generate a "hypothetical" document in response to a query. This hypothetical document is then used to retrieve actual, similar documents, improving relevance. |
|
||||
| **Late Chunking** <br /> <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/Advanced_RAG_Late_Chunking/Late_Chunking_(Chunked_Pooling).ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab" /></a>[View on GitHub](https://github.com/lancedb/vectordb-recipes/tree/main/examples/Advanced_RAG_Late_Chunking) | An advanced RAG method where documents are retrieved first, and then chunking is performed on the retrieved documents just before synthesis. This helps maintain context that might be lost with pre-chunking. |
|
||||
| **Agentic RAG** <br /> <a href="https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/tutorials/Agentic_RAG/main.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab" /></a>[View on GitHub](https://github.com/lancedb/vectordb-recipes/tree/main/tutorials/Agentic_RAG) | This tutorial demonstrates how to build a RAG system where multiple AI agents collaborate to retrieve information and generate answers, leading to more robust and intelligent applications. |
|
||||
@@ -0,0 +1,71 @@
|
||||
---
|
||||
title: "Multimodal Agent"
|
||||
sidebarTitle: "Multimodal agent"
|
||||
description: "Build an AI agent that understands both text and images to help users find recipes using LanceDB and PydanticAI"
|
||||
---
|
||||
|
||||
|
||||
[](https://colab.research.google.com/drive/1pxavAGoXa-KSh_4HxNpvP2AjHPcIRpbq?usp=sharing)
|
||||
|
||||
Ever wanted to combine the power of text and images in a single AI agent? In this tutorial,
|
||||
you'll build an agent that can understand both text and images to help users discover recipes that are relevant to them. The approach shown combines LanceDB's multimodal capabilities with [Pydantic AI](https://ai.pydantic.dev/) for the agentic workflow.
|
||||
|
||||
## Key Technologies {#key-technologies}
|
||||
|
||||
- **LanceDB**: Embedded retrieval library and multimodal lakehouse for efficient storage and retrieval
|
||||
- **PydanticAI**: Modern AI agent framework with type safety
|
||||
- **Sentence Transformers**: Text embeddings for semantic search
|
||||
- **CLIP**: Vision-language model for image understanding
|
||||
- **Streamlit**: Interactive web application framework
|
||||
|
||||
## Tutorial Overview {#tutorial-overview}
|
||||
|
||||
### Option 1: Notebook {#option-1-notebook}
|
||||
The notebook shows how to work through the steps and prepare a small sample recipe dataset, generate both text and image
|
||||
embeddings, store everything efficiently in LanceDB, and then build a PydanticAI agent with custom tools to
|
||||
query it. You'll finish by testing the agent against a few example questions to see the full multimodal flow
|
||||
end to end.
|
||||
|
||||
<Card title="Notebook" href="https://colab.research.google.com/drive/1pxavAGoXa-KSh_4HxNpvP2AjHPcIRpbq?usp=sharing">
|
||||
This simple tutorial provides a step-by-step workflow with a small demo dataset of 4 examples.
|
||||
No local setup required - just click and start learning about multimodal agents.
|
||||
[](https://colab.research.google.com/drive/1pxavAGoXa-KSh_4HxNpvP2AjHPcIRpbq?usp=sharing)
|
||||
</Card>
|
||||
|
||||
### Option 2: Demo Application (Local Setup) {#option-2-demo-application-local-setup}
|
||||
The demo application is the full codebase: you'll download and process a real recipe dataset with thousands
|
||||
of items, run a Streamlit chat interface that supports image upload, and follow a structure that includes
|
||||
production-minded touches like error handling, logging, and monitoring. Everything you need to deploy is
|
||||
included.
|
||||
|
||||
<Card icon="download" title="Download Tutorial Files" href="https://github.com/lancedb/vectordb-recipes/tree/main/applications/multimodal-recipe-agent">
|
||||
Download the files for the full demo application here.
|
||||
</Card>
|
||||
|
||||
### Dataset Information {#dataset-information}
|
||||
- **Source**: [Kaggle Recipe Dataset](https://www.kaggle.com/datasets/pes12017000148/food-ingredients-and-recipe-dataset-with-images)
|
||||
- **Size**: Thousands of recipes with images
|
||||
- **Format**: CSV file with recipe data and image references
|
||||
|
||||
### Setup {#setup}
|
||||
|
||||
<CodeGroup>
|
||||
```bash bash icon="code"
|
||||
# 1. Extract the downloaded files to a folder
|
||||
# 2. Navigate to the folder in terminal
|
||||
cd multimodal-recipe-agent
|
||||
|
||||
# 3. Install dependencies with uv
|
||||
uv sync
|
||||
|
||||
# 4. Download the Kaggle dataset
|
||||
# Visit: https://www.kaggle.com/datasets/pes12017000148/food-ingredients-and-recipe-dataset-with-images
|
||||
# Extract recipes.csv to the data/ folder
|
||||
|
||||
# 5. Import the dataset
|
||||
uv run python import.py
|
||||
|
||||
# 6. Run the complete Streamlit chat application
|
||||
uv run streamlit run app.py
|
||||
```
|
||||
</CodeGroup>
|
||||
@@ -0,0 +1,175 @@
|
||||
---
|
||||
title: "NVIDIA RAG Blueprint with LanceDB"
|
||||
sidebarTitle: "NVIDIA RAG Blueprint"
|
||||
description: "Use LanceDB as the retrieval layer for NVIDIA RAG Blueprint with a Docker-first, retrieval-only reference integration."
|
||||
---
|
||||
|
||||
## What this tutorial shows {#what-this-tutorial-shows}
|
||||
|
||||
If you are using [NVIDIA RAG Blueprints](https://build.nvidia.com/blueprints) and want to evaluate LanceDB in that stack, this tutorial gives you a concrete starting point. It shows how to use LanceDB as the retrieval layer for a Docker-based NVIDIA RAG deployment with a small, script-driven reference integration where LanceDB OSS is embedded directly in the NVIDIA containers, the collection is prepared ahead of time, and the RAG server retrieves from it for search and generation. The example is intentionally retrieval-only, but it also includes hybrid search and reranker selection so you can see how LanceDB fits into a realistic NVIDIA retrieval workflow.
|
||||
|
||||
<Tip>
|
||||
The runnable example for this tutorial lives in the
|
||||
[VectorDB recipes repository](https://github.com/lancedb/vectordb-recipes/tree/main/examples/nvidia-rag-blueprint-lancedb).
|
||||
</Tip>
|
||||
|
||||
|
||||
## How NVIDIA organizes vector databases {#how-nvidia-organizes-vector-databases}
|
||||
|
||||
NVIDIA's [RAG Blueprint documentation](https://docs.nvidia.com/rag/latest/readme.html) effectively describes three different patterns for vector database support.
|
||||
1. There are built-in backends such as Milvus, where NVIDIA already owns both ingestion and
|
||||
retrieval.
|
||||
2. There are built-in alternatives such as Elasticsearch, where NVIDIA still owns
|
||||
the end-to-end flow but switches the backend through configuration.
|
||||
3. Then, there is the custom vector database path, where you implement a `VDBRag` backend yourself and register it in NVIDIA's
|
||||
factory.
|
||||
|
||||
The LanceDB example shown below fits into the third category. More specifically, it follows NVIDIA's
|
||||
**retrieval-only** custom backend path: the data is prepared in LanceDB ahead of time, and NVIDIA
|
||||
RAG Blueprint is then pointed at that existing collection for search and generation. It does not
|
||||
yet teach NVIDIA's ingestor how to write new documents into LanceDB automatically.
|
||||
|
||||
## Deployment model {#deployment-model}
|
||||
|
||||
This reference integration uses **LanceDB OSS as an embedded retrieval library**, not as a separate
|
||||
database service. In practice, `APP_VECTORSTORE_NAME` is set to `lancedb`, `APP_VECTORSTORE_URL`
|
||||
points to a local filesystem path inside the NVIDIA containers, the LanceDB collection is prepared
|
||||
ahead of time, and the NVIDIA RAG server loads the LanceDB adapter to retrieve directly from that
|
||||
local dataset.
|
||||
|
||||
## What the recipe contains {#what-the-recipe-contains}
|
||||
|
||||
The recipe at
|
||||
[`examples/nvidia-rag-blueprint-lancedb`](https://github.com/lancedb/vectordb-recipes/tree/main/examples/nvidia-rag-blueprint-lancedb)
|
||||
is organized around a small number of practical pieces. The data-prep script builds a demo
|
||||
LanceDB collection from scratch, generates embeddings through the LanceDB embedding registry, and
|
||||
creates a full-text index so hybrid retrieval works immediately. The adapter file shows the
|
||||
retrieval-only integration point for NVIDIA RAG Blueprint, while the Docker override and NVIDIA
|
||||
change guide show the minimal configuration and source changes needed to run the example against
|
||||
NVIDIA's containers.
|
||||
|
||||
## End-to-end flow {#end-to-end-flow}
|
||||
|
||||
### 1\. Prepare the LanceDB collection {#1-prepare-the-lancedb-collection}
|
||||
|
||||
From the [recipe directory](https://github.com/lancedb/vectordb-recipes/tree/main/examples/nvidia-rag-blueprint-lancedb):
|
||||
|
||||
```bash
|
||||
uv sync
|
||||
uv run prepare_lancedb.py --embedder demo-keyword --reranker mrr
|
||||
```
|
||||
|
||||
That script creates:
|
||||
|
||||
- a local LanceDB dataset under `data/`
|
||||
- a collection named `nvidia_blueprint_demo`
|
||||
- automatic embeddings generated at ingest time
|
||||
- an FTS index for hybrid search
|
||||
|
||||
The default embedder is an offline demo embedder so the example stays easy to run. If you want a
|
||||
more realistic setup, the same script can switch to a sentence-transformers embedder.
|
||||
|
||||
### 2\. Patch the NVIDIA blueprint {#2-patch-the-nvidia-blueprint}
|
||||
|
||||
Follow the instructions in the recipe's
|
||||
[`nvidia_blueprint_changes.md`](https://github.com/lancedb/vectordb-recipes/tree/main/examples/nvidia-rag-blueprint-lancedb/nvidia_blueprint_changes.md).
|
||||
The essential changes are:
|
||||
|
||||
- add LanceDB dependencies to the NVIDIA environment
|
||||
- copy `lancedb_vdb.py` into the NVIDIA source tree
|
||||
- register the `lancedb` branch in NVIDIA's VDB factory
|
||||
|
||||
NVIDIA's [RAG blueprint documentation](https://docs.nvidia.com/rag/latest/readme.html) and custom-VDB guide provide useful background if you want more context before applying the LanceDB-specific changes.
|
||||
|
||||
### 3\. Start the Docker deployment {#3-start-the-docker-deployment}
|
||||
|
||||
Set the absolute path to the recipe directory:
|
||||
|
||||
```bash
|
||||
export LANCEDB_RECIPE_DIR=/absolute/path/to/vectordb-recipes/examples/nvidia-rag-blueprint-lancedb
|
||||
```
|
||||
|
||||
Then from the NVIDIA repo root:
|
||||
|
||||
```bash
|
||||
docker compose \
|
||||
-f deploy/compose/docker-compose-rag-server.yaml \
|
||||
-f "$LANCEDB_RECIPE_DIR"/docker-compose.override.yml \
|
||||
up -d --build
|
||||
|
||||
docker compose \
|
||||
-f deploy/compose/docker-compose-ingestor-server.yaml \
|
||||
-f "$LANCEDB_RECIPE_DIR"/docker-compose.override.yml \
|
||||
up -d --build
|
||||
```
|
||||
|
||||
The key environment values are:
|
||||
|
||||
- `APP_VECTORSTORE_NAME=lancedb`
|
||||
- `APP_VECTORSTORE_URL=/opt/lancedb-recipe/data`
|
||||
- `COLLECTION_NAME=nvidia_blueprint_demo`
|
||||
- `APP_VECTORSTORE_SEARCHTYPE=hybrid`
|
||||
- `LANCEDB_RERANKER=mrr`
|
||||
|
||||
## Verifying the integration {#verifying-the-integration}
|
||||
|
||||
### Search {#search}
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8081/v1/search \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"query": "How do I replace Milvus in the NVIDIA RAG blueprint with LanceDB?",
|
||||
"use_knowledge_base": true,
|
||||
"collection_names": ["nvidia_blueprint_demo"],
|
||||
"vdb_top_k": 3,
|
||||
"reranker_top_k": 0
|
||||
}'
|
||||
```
|
||||
|
||||
### Generate {#generate}
|
||||
|
||||
```bash
|
||||
curl -N -X POST http://localhost:8081/v1/generate \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"messages": [{"role":"user","content":"Summarize the LanceDB integration approach."}],
|
||||
"use_knowledge_base": true,
|
||||
"collection_names": ["nvidia_blueprint_demo"],
|
||||
"vdb_top_k": 3,
|
||||
"reranker_top_k": 0
|
||||
}'
|
||||
```
|
||||
|
||||
## Hybrid retrieval and rerankers {#hybrid-retrieval-and-rerankers}
|
||||
|
||||
This example is meant to prove more than a trivial vector lookup.
|
||||
|
||||
- LanceDB hybrid retrieval combines vector search with full-text search
|
||||
- the recipe creates the FTS index as part of dataset prep
|
||||
- the adapter supports `RRFReranker`, `MRRReranker`, and `CrossEncoderReranker`
|
||||
- the default example uses `MRRReranker`, not a plain weighted linear combination
|
||||
|
||||
That matters for NVIDIA partner workloads because product names, storage platforms, and technical
|
||||
jargon often need exact lexical matching as well as semantic retrieval.
|
||||
|
||||
## How this can be extended {#how-this-can-be-extended}
|
||||
|
||||
The current example follows NVIDIA's **custom retrieval-only backend** path. In practice, that
|
||||
means the LanceDB collection is created ahead of time and NVIDIA RAG Blueprint is then pointed at
|
||||
that existing collection for search and generation. The sample data in `prepare_lancedb.py` exists
|
||||
only to make that flow runnable end to end: it creates a small local collection, inserts a few
|
||||
documents, generates embeddings, and builds an FTS index so the NVIDIA side has something real to
|
||||
query.
|
||||
|
||||
A fuller integration is possible. NVIDIA's custom `VDBRag` interface also supports the pattern used
|
||||
by built-in backends such as Milvus and Elasticsearch, where NVIDIA owns both ingestion and
|
||||
retrieval. To make LanceDB work that way, a complete LanceDB backend would need to implement the
|
||||
ingestion methods NVIDIA documents, especially `create_collection` and `write_to_index`, along with
|
||||
the retrieval and collection-management methods expected by the rest of the stack.
|
||||
|
||||
The open work is in defining how NVIDIA's ingestor should write
|
||||
records into LanceDB, how that storage is shared between the ingestor and the RAG server, and how
|
||||
document and collection metadata should be exposed so the broader NVIDIA APIs behave correctly.
|
||||
Until those pieces exist, this example should be read as: prepare LanceDB first, then let NVIDIA
|
||||
retrieve from it.
|
||||
@@ -0,0 +1,179 @@
|
||||
---
|
||||
title: "Time-Travel RAG with versioned data"
|
||||
sidebarTitle: "Time-travel RAG"
|
||||
description: "Learn how to build production-ready RAG systems with LanceDB's time-travel capabilities for regulatory compliance and audit trails."
|
||||
---
|
||||
|
||||
<Tip>
|
||||
All the scripts and code for this tutorial are available in the
|
||||
[vectorDB recipes](https://github.com/lancedb/vectordb-recipes/tree/main/examples/time-travel-rag) repository.
|
||||
</Tip>
|
||||
|
||||
## Use case: Financial services regulatory knowledge base {#use-case-financial-services-regulatory-knowledge-base}
|
||||
|
||||
Imagine you're a major investment bank. Your team is tasked with building a critical Retrieval-Augmented Generation (RAG) system. This system must provide instant, accurate answers to compliance officers about ever-changing financial regulations. A wrong or out-of-date answer isn't just an inconvenience—it could lead to multi-million dollar fines, reputational damage, and regulatory audits.
|
||||
|
||||
Your knowledge base is a living entity, constantly evolving with:
|
||||
|
||||
* Daily regulatory updates from government bodies.
|
||||
* New internal policy documents and interpretations.
|
||||
* A/B testing of different embedding models and text chunking strategies to improve accuracy.
|
||||
|
||||
This dynamic environment creates a series of high-stakes challenges that traditional
|
||||
vector databases are ill-equipped to handle.
|
||||
|
||||
## Pain points solved by LanceDB {#pain-points-solved-by-lancedb}
|
||||
|
||||
1. "Our RAG gave different answers yesterday versus today. Which version was used in the official compliance report?" Without versioning, you can't prove what the AI knew at a specific point in time, making audits impossible.
|
||||
|
||||
2. "The new embedding model we deployed corrupted half the vectors. Can we instantly roll back our 10TB dataset?" With traditional systems, a rollback means a painful, hours-long (or days-long) process of re-indexing from a backup, leading to significant downtime.
|
||||
|
||||
3. "Regulators want to audit an AI-assisted decision from three months ago. How can we prove what data the model had access to at that exact moment?" Reproducibility is key for compliance. You must be able to reconstruct the exact state of the knowledge base for any historical query.
|
||||
|
||||
4. "We need to A/B test a new chunking strategy, but we can't disrupt the production system or duplicate the entire dataset." Experimentation is vital for improvement, but it can't come at the cost of production stability or a massive infrastructure bill.
|
||||
|
||||
LanceDB's [zero-cost data evolution](/tables/schema) and [time-travel capabilities](https://docs.lancedb.com/tables/versioning) directly address these critical enterprise pain points, providing the foundation for a reliable, auditable, and production-ready RAG system.
|
||||
|
||||
## Dataset: The U.S. Federal Register {#dataset-the-us-federal-register}
|
||||
|
||||
To make this use case realistic, we'll use a perfect real-world dataset: The U.S. Federal Register, the official daily journal of the United States Government.
|
||||
|
||||
It contains all new rules, proposed rules, and notices from federal agencies. It is the canonical source for regulatory changes, and it's updated every business day. It even has a public API, allowing us to simulate the real-time ingestion of new documents.
|
||||
|
||||
An example output of the workflow defined in main.py
|
||||
is shown below.
|
||||
|
||||
```bash main.py expandable
|
||||
--- Initializing Database Environment ---
|
||||
Removed old database at ./lancedb
|
||||
Loading embedding model: all-MiniLM-L6-v2...
|
||||
|
||||
--- STEP 1: Initial Data Ingestion ---
|
||||
|
||||
Fetching 500 documents for publication date: 2024-08-19...
|
||||
Successfully fetched 86 documents.
|
||||
Embedding 86 documents...
|
||||
Batches: 100%|███████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 7.12it/s]
|
||||
Successfully embedded 86 documents.
|
||||
|
||||
Creating table 'federal_register'...
|
||||
✅ Table 'federal_register' created. Version: 1, Rows: 86
|
||||
|
||||
--- STEP 2: Simulating Sequential Daily Updates ---
|
||||
|
||||
Fetching 500 documents for publication date: 2024-08-20...
|
||||
Successfully fetched 102 documents.
|
||||
Embedding 102 documents...
|
||||
Batches: 100%|███████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 10.66it/s]
|
||||
Successfully embedded 102 documents.
|
||||
✅ Data added to 'federal_register'. New Version: 2, Total Rows: 188
|
||||
|
||||
Fetching 500 documents for publication date: 2024-08-21...
|
||||
Successfully fetched 114 documents.
|
||||
Embedding 114 documents...
|
||||
Batches: 100%|███████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 10.19it/s]
|
||||
Successfully embedded 114 documents.
|
||||
✅ Data added to 'federal_register'. New Version: 3, Total Rows: 302
|
||||
|
||||
|
||||
========================================================
|
||||
= PART 1: AUDITING KNOWLEDGE BASE ACROSS TIME =
|
||||
========================================================
|
||||
|
||||
Running audit for query: 'cybersecurity reporting requirements for public companies'
|
||||
|
||||
Attempting to open 'federal_register' and checkout version 1...
|
||||
✅ Successfully checked out Version 1 of 'federal_register'. Total rows: 86
|
||||
Querying table 'federal_register' (Version 1)...
|
||||
|
||||
--- Top Result for Version: V1 (all-MiniLM-L6-v2) ---
|
||||
📄 Title: Public Company Accounting Oversight Board; Extension of Approval Periods for Proposed Rules on a Firm's System of Quality Control and Related Amendments to PCAOB Standards, Proposed Rules on Amendments Related to Aspects of Designing and Performing Audit Procedures That Involve Technology-Assisted Analysis of Information in Electronic Form, and Proposed Rules on Amendment to PCAOB Rule 3502 Governing Contributory Liability
|
||||
🗓️ Date: 2024-08-19
|
||||
📏 Distance: 1.1667
|
||||
📝 Abstract:
|
||||
[No abstract available for this document]
|
||||
--------------------------------------
|
||||
|
||||
Attempting to open 'federal_register' and checkout version 2...
|
||||
✅ Successfully checked out Version 2 of 'federal_register'. Total rows: 188
|
||||
Querying table 'federal_register' (Version 2)...
|
||||
|
||||
--- Top Result for Version: V2 (all-MiniLM-L6-v2) ---
|
||||
📄 Title: Information Collection Being Reviewed by the Federal Communications Commission Under Delegated Authority
|
||||
🗓️ Date: 2024-08-20
|
||||
📏 Distance: 1.1436
|
||||
📝 Abstract:
|
||||
As part of its continuing effort to reduce paperwork burdens, and as required by the Paperwork
|
||||
Reduction Act (PRA) of 1995, the Federal Communications Commission (FCC or the Commission) invites
|
||||
the general public and other Federal agencies to take this opportunity to comment on the following
|
||||
information collection. Comments are requested concerning: whether the proposed collection of
|
||||
information is necessary for the proper performance of the functions of the Commission, including
|
||||
whether the information shall have practical utility; the accuracy of the Commission's burden
|
||||
estimate; ways to enhance the quality, utility, and clarity of the information collected; ways to
|
||||
minimize the burden of the collection of information on the respondents, including the use of
|
||||
automated collection techniques or other forms of information technology; and ways to further reduce
|
||||
the information collection burden on small business concerns with fewer than 25 employees. The FCC
|
||||
may not conduct or sponsor a collection of information unless it displays a currently valid control
|
||||
number. No person shall be subject to any penalty for failing to comply with a collection of
|
||||
information subject to the PRA that does not display a valid Office of Management and Budget (OMB)
|
||||
control number.
|
||||
--------------------------------------
|
||||
|
||||
Attempting to open 'federal_register' and checkout version 3...
|
||||
✅ Successfully checked out Version 3 of 'federal_register'. Total rows: 302
|
||||
Querying table 'federal_register' (Version 3)...
|
||||
|
||||
--- Top Result for Version: V3 (all-MiniLM-L6-v2) ---
|
||||
📄 Title: Equipment, Systems, and Network Information Security Protection
|
||||
🗓️ Date: 2024-08-21
|
||||
📏 Distance: 1.0942
|
||||
📝 Abstract:
|
||||
This proposed rulemaking would impose new design standards to address cybersecurity threats for
|
||||
transport category airplanes, engines, and propellers. The intended effect of this proposed action
|
||||
is to standardize the FAA's criteria for addressing cybersecurity threats, reducing certification
|
||||
costs and time while maintaining the same level of safety provided by current special conditions.
|
||||
--------------------------------------
|
||||
|
||||
✅ Date-based audit complete. Results show how knowledge evolves over time. This demonstrates LanceDB's powerful [versioning capabilities](/tutorials/tables/consistency#versioning) for maintaining audit trails.
|
||||
|
||||
|
||||
=============================================================
|
||||
= PART 2: A/B TESTING DIFFERENT EMBEDDING MODELS =
|
||||
=============================================================
|
||||
Loading embedding model: all-mpnet-base-v2...
|
||||
Embedding 302 documents...
|
||||
Batches: 100%|█████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:03<00:00, 2.55it/s]
|
||||
Successfully embedded 302 documents.
|
||||
|
||||
Creating table 'federal_register_experimental'...
|
||||
✅ Table 'federal_register_experimental' created. Version: 1, Rows: 302
|
||||
|
||||
Comparing search results for the same data with different models:
|
||||
Querying table 'federal_register' (Version 3)...
|
||||
|
||||
--- Top Result for Version: Latest Prod V3 (all-MiniLM-L6-v2) ---
|
||||
📄 Title: Equipment, Systems, and Network Information Security Protection
|
||||
🗓️ Date: 2024-08-21
|
||||
📏 Distance: 1.0942
|
||||
📝 Abstract:
|
||||
This proposed rulemaking would impose new design standards to address cybersecurity threats for
|
||||
transport category airplanes, engines, and propellers. The intended effect of this proposed action
|
||||
is to standardize the FAA's criteria for addressing cybersecurity threats, reducing certification
|
||||
costs and time while maintaining the same level of safety provided by current special conditions.
|
||||
--------------------------------------
|
||||
Querying table 'federal_register_experimental' (Version 1)...
|
||||
|
||||
--- Top Result for Version: Experimental (all-mpnet-base-v2) ---
|
||||
📄 Title: Commission Information Collection Activities (FERC-725B); Comment Request; Extension
|
||||
🗓️ Date: 2024-08-20
|
||||
📏 Distance: 1.0827
|
||||
📝 Abstract:
|
||||
In compliance with the requirements of the Paperwork Reduction Act of 1995, the Federal Energy
|
||||
Regulatory Commission (Commission or FERC) is soliciting public comment on the currently approved
|
||||
information collection, FERC-725B, Mandatory Reliability Standards, Critical Infrastructure
|
||||
Protection (CIP) (Update for CIP-012-1 to version CIP-012-02) Cyber Security--Communications between
|
||||
Control Centers. The 60-day notice comment period ended on July 23, 2024, with no comments received.
|
||||
--------------------------------------
|
||||
|
||||
✅ A/B test complete. Notice the difference in relevance (distance score) between models. This showcases how LanceDB enables [experimentation with different embedding models](/docs/embeddings/) without disrupting production systems.
|
||||
```
|
||||
Reference in New Issue
Block a user