From f2e398983187e240c46c8536b2c8968e0c457744 Mon Sep 17 00:00:00 2001 From: fzowl <160063452+fzowl@users.noreply.github.com> Date: Mon, 18 Nov 2024 23:34:16 +0100 Subject: [PATCH] docs: voyageai embedding in the index (#1813) The code to support VoyageAI embedding and rerank models was added in the https://github.com/lancedb/lancedb/pull/1799 PR. Some of the documentation changes was also made, here adding the VoyageAI embedding doc link to the index page. These are my first PRs in lancedb and while i checked the documentation/code structure, i might missed something important. Please let me know if any changes required! --- .../text_embedding_functions/voyageai_embedding.md | 2 +- docs/src/embeddings/default_embedding_functions.md | 2 ++ docs/src/reranking/index.md | 2 ++ 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/src/embeddings/available_embedding_models/text_embedding_functions/voyageai_embedding.md b/docs/src/embeddings/available_embedding_models/text_embedding_functions/voyageai_embedding.md index 41a6be31..beb0b7f7 100644 --- a/docs/src/embeddings/available_embedding_models/text_embedding_functions/voyageai_embedding.md +++ b/docs/src/embeddings/available_embedding_models/text_embedding_functions/voyageai_embedding.md @@ -20,7 +20,7 @@ Supported parameters (to be passed in `create` method) are: | Parameter | Type | Default Value | Description | |---|---|--------|---------| -| `name` | `str` | `"voyage-3"` | The model ID of the model to use. Supported base models for Text Embeddings: voyage-3, voyage-3-lite, voyage-finance-2, voyage-multilingual-2, voyage-law-2, voyage-code-2 | +| `name` | `str` | `None` | The model ID of the model to use. Supported base models for Text Embeddings: voyage-3, voyage-3-lite, voyage-finance-2, voyage-multilingual-2, voyage-law-2, voyage-code-2 | | `input_type` | `str` | `None` | Type of the input text. Default to None. Other options: query, document. | | `truncation` | `bool` | `True` | Whether to truncate the input texts to fit within the context length. | diff --git a/docs/src/embeddings/default_embedding_functions.md b/docs/src/embeddings/default_embedding_functions.md index 5457dc9f..5d99ec7e 100644 --- a/docs/src/embeddings/default_embedding_functions.md +++ b/docs/src/embeddings/default_embedding_functions.md @@ -53,6 +53,7 @@ These functions are registered by default to handle text embeddings. | [**Jina Embeddings**](available_embedding_models/text_embedding_functions/jina_embedding.md "jina") | 🔗 World-class embedding models to improve your search and RAG systems. You will need **jina api key**. | [Jina Icon](available_embedding_models/text_embedding_functions/jina_embedding.md) | | [ **AWS Bedrock Functions**](available_embedding_models/text_embedding_functions/aws_bedrock_embedding.md "bedrock-text") | ☁️ AWS Bedrock supports multiple base models for generating text embeddings. You need to setup the AWS credentials to use this embedding function. | [AWS Bedrock Icon](available_embedding_models/text_embedding_functions/aws_bedrock_embedding.md) | | [**IBM Watsonx.ai**](available_embedding_models/text_embedding_functions/ibm_watsonx_ai_embedding.md "watsonx") | 💡 Generate text embeddings using IBM's watsonx.ai platform. **Note**: watsonx.ai library is an optional dependency. | [Watsonx Icon](available_embedding_models/text_embedding_functions/ibm_watsonx_ai_embedding.md) | +| [**VoyageAI Embeddings**](available_embedding_models/text_embedding_functions/voyageai_embedding.md "voyageai") | 🌕 Voyage AI provides cutting-edge embedding and rerankers. This will help you get started with **VoyageAI** embedding models using LanceDB. Using voyageai API requires voyageai package. Install it via `pip`. | [VoyageAI Icon](available_embedding_models/text_embedding_functions/voyageai_embedding.md) | @@ -66,6 +67,7 @@ These functions are registered by default to handle text embeddings. [jina-key]: "jina" [aws-key]: "bedrock-text" [watsonx-key]: "watsonx" +[voyageai-key]: "voyageai" ## Multi-modal Embedding Functions🖼️ diff --git a/docs/src/reranking/index.md b/docs/src/reranking/index.md index 746c5d4e..2e880913 100644 --- a/docs/src/reranking/index.md +++ b/docs/src/reranking/index.md @@ -9,6 +9,7 @@ LanceDB comes with some built-in rerankers. Some of the rerankers that are avail | `CrossEncoderReranker` | Uses a cross-encoder model to rerank search results | Vector, FTS, Hybrid | | `ColbertReranker` | Uses a colbert model to rerank search results | Vector, FTS, Hybrid | | `OpenaiReranker`(Experimental) | Uses OpenAI's chat model to rerank search results | Vector, FTS, Hybrid | +| `VoyageAIReranker` | Uses voyageai Reranker API to rerank results | Vector, FTS, Hybrid | ## Using a Reranker @@ -73,6 +74,7 @@ LanceDB comes with some built-in rerankers. Here are some of the rerankers that - [Jina Reranker](./jina.md) - [AnswerDotAI Rerankers](./answerdotai.md) - [Reciprocal Rank Fusion Reranker](./rrf.md) +- [VoyageAI Reranker](./voyageai.md) ## Creating Custom Rerankers