diff --git a/docs/src/examples/examples_python.md b/docs/src/examples/examples_python.md index 2c7d17d6..6ffe972f 100644 --- a/docs/src/examples/examples_python.md +++ b/docs/src/examples/examples_python.md @@ -10,13 +10,13 @@ Explore applied examples available as Colab notebooks or Python scripts to integ | Explore | Description | |----------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Build from Scratch with LanceDB πŸ› οΈπŸš€](python_examples/build_from_scratch.md) | Start building your GenAI applications from the ground up using LanceDB's efficient vector-based document retrieval capabilities! Get started quickly with a solid foundation. | -| [Multimodal Search with LanceDB πŸ€Ήβ€β™‚οΈπŸ”](python_examples/multimodal.md) | Combine text and image queries to find the most relevant results using LanceDB’s multimodal capabilities. Leverage the efficient vector-based similarity search. | -| [RAG: Revolutionize Information Retrieval with LanceDB πŸ”“πŸ§](python_examples/rag.md) | Build RAG (Retrieval-Augmented Generation) with LanceDB for efficient vector-based information retrieval and more accurate responses from AI. | -| [Vector Search: Unlock Efficient Document Retrieval πŸ”“πŸ‘€](python_examples/vector_search.md) | Use LanceDB's vector search capabilities to perform efficient and accurate similarity searches, enabling rapid discovery and retrieval of relevant documents in Large datasets. | -| [Chatbot Application with LanceDB πŸ€–](python_examples/chatbot.md) | Create chatbots that retrieves relevant context for coherent and context-aware replies, enhancing user experience through advanced conversational AI. | -| [Evaluation: Assessing Text Performance with Precision πŸ“ŠπŸ’‘](python_examples/evaluations.md) | Develop evaluation applications that allows you to input reference and candidate texts to measure their performance across various metrics. | -| [AI Agents: Intelligent Collaboration πŸ€–](python_examples/aiagent.md) | Enable AI agents to communicate and collaborate efficiently through dense vector representations, achieving shared goals seamlessly. | -| [Recommender Systems: Personalized Discovery πŸΏπŸ“Ί](python_examples/recommendersystem.md) | Deliver personalized experiences by efficiently storing and querying item embeddings with LanceDB's powerful vector database capabilities. | -| **Miscellaneous Examples🌟** | Find other unique examples and creative solutions using LanceDB, showcasing the flexibility and broad applicability of the platform. | +| [**Build from Scratch with LanceDB** πŸ› οΈπŸš€](python_examples/build_from_scratch.md) | Start building your **GenAI applications** from the **ground up** using **LanceDB's** efficient vector-based document retrieval capabilities! Get started quickly with a solid foundation. | +| [**Multimodal Search with LanceDB** πŸ€Ήβ€β™‚οΈπŸ”](python_examples/multimodal.md) | Combine **text** and **image queries** to find the most relevant results using **LanceDB’s multimodal** capabilities. Leverage the efficient vector-based similarity search. | +| [**RAG (Retrieval-Augmented Generation) with LanceDB** πŸ”“πŸ§](python_examples/rag.md) | Build RAG (Retrieval-Augmented Generation) with **LanceDB** for efficient **vector-based information retrieval** and more accurate responses from AI. | +| [**Vector Search: Efficient Retrieval** πŸ”“πŸ‘€](python_examples/vector_search.md) | Use **LanceDB's** vector search capabilities to perform efficient and accurate **similarity searches**, enabling rapid discovery and retrieval of relevant documents in Large datasets. | +| [**Chatbot applications with LanceDB** πŸ€–](python_examples/chatbot.md) | Create **chatbots** that retrieves relevant context for **coherent and context-aware replies**, enhancing user experience through advanced conversational AI. | +| [**Evaluation: Assessing Text Performance with Precision** πŸ“ŠπŸ’‘](python_examples/evaluations.md) | Develop **evaluation** applications that allows you to input reference and candidate texts to **measure** their performance across various metrics. | +| [**AI Agents: Intelligent Collaboration** πŸ€–](python_examples/aiagent.md) | Enable **AI agents** to communicate and collaborate efficiently through dense vector representations, achieving shared goals seamlessly. | +| [**Recommender Systems: Personalized Discovery** πŸΏπŸ“Ί](python_examples/recommendersystem.md) | Deliver **personalized experiences** by efficiently storing and querying item embeddings with **LanceDB's** powerful vector database capabilities. | +| **Miscellaneous Examples🌟** | Find other **unique examples** and **creative solutions** using **LanceDB**, showcasing the flexibility and broad applicability of the platform. | diff --git a/docs/src/examples/python_examples/aiagent.md b/docs/src/examples/python_examples/aiagent.md index 12b624ae..bcb2eb20 100644 --- a/docs/src/examples/python_examples/aiagent.md +++ b/docs/src/examples/python_examples/aiagent.md @@ -1,15 +1,15 @@ # AI Agents: Intelligent CollaborationπŸ€– -Think of a platformπŸ’» where AI AgentsπŸ€– can seamlessly exchange information, coordinate over tasks, and achieve shared targets with great efficiencyπŸ“ˆπŸš€. +Think of a platform where AI Agents can seamlessly exchange information, coordinate over tasks, and achieve shared targets with great efficiencyπŸ’»πŸ“ˆ. ## Vector-Based Coordination: The Technical Advantage -Leveraging LanceDB's vector-based capabilities, our coordination application enables AI agents to communicate and collaborate through dense vector representations πŸ€–. AI agents can exchange information, coordinate on a task or work towards a common goal, just by giving queriesπŸ“. +Leveraging LanceDB's vector-based capabilities, we can enable **AI agents πŸ€–** to communicate and collaborate through dense vector representations. AI agents can exchange information, coordinate on a task or work towards a common goal, just by giving queriesπŸ“. | **AI Agents** | **Description** | **Links** | |:--------------|:----------------|:----------| -| **AI Agents: Reducing HallucinationtπŸ“Š** | πŸ€–πŸ’‘ Reduce AI hallucinations using Critique-Based Contexting! Learn by Simplifying and Automating tedious workflows by going through fitness trainer agent example.πŸ’ͺ | [![Github](../../assets/github.svg)][hullucination_github]
[![Open In Collab](../../assets/colab.svg)][hullucination_colab]
[![Python](../../assets/python.svg)][hullucination_python]
[![Ghost](../../assets/ghost.svg)][hullucination_ghost] | -| **AI Trends Searcher: CrewAIπŸ”οΈ** | πŸ”οΈ Learn about CrewAI Agents ! Utilize the features of CrewAI - Role-based Agents, Task Management, and Inter-agent Delegation ! Make AI agents work together to do tricky stuff 😺| [![Github](../../assets/github.svg)][trend_github]
[![Open In Collab](../../assets/colab.svg)][trend_colab]
[![Ghost](../../assets/ghost.svg)][trend_ghost] | -| **SuperAgent AutogenπŸ€–** | πŸ’» AI interactions with the Super Agent! Integrating Autogen, LanceDB, LangChain, LiteLLM, and Ollama to create AI agent that excels in understanding and processing complex queries.πŸ€– | [![Github](../../assets/github.svg)][superagent_github]
[![Open In Collab](../../assets/colab.svg)][superagent_colab] | +| **AI Agents: Reducing HallucinationtπŸ“Š** | πŸ€–πŸ’‘ **Reduce AI hallucinations** using Critique-Based Contexting! Learn by Simplifying and Automating tedious workflows by going through fitness trainer agent example.πŸ’ͺ | [![Github](../../assets/github.svg)][hullucination_github]
[![Open In Collab](../../assets/colab.svg)][hullucination_colab]
[![Python](../../assets/python.svg)][hullucination_python]
[![Ghost](../../assets/ghost.svg)][hullucination_ghost] | +| **AI Trends Searcher: CrewAIπŸ”οΈ** | πŸ”οΈ Learn about **CrewAI Agents** ! Utilize the features of CrewAI - Role-based Agents, Task Management, and Inter-agent Delegation ! Make AI agents work together to do tricky stuff 😺| [![Github](../../assets/github.svg)][trend_github]
[![Open In Collab](../../assets/colab.svg)][trend_colab]
[![Ghost](../../assets/ghost.svg)][trend_ghost] | +| **SuperAgent AutogenπŸ€–** | πŸ’» AI interactions with the Super Agent! Integrating **Autogen**, **LanceDB**, **LangChain**, **LiteLLM**, and **Ollama** to create AI agent that excels in understanding and processing complex queries.πŸ€– | [![Github](../../assets/github.svg)][superagent_github]
[![Open In Collab](../../assets/colab.svg)][superagent_colab] | [hullucination_github]: https://github.com/lancedb/vectordb-recipes/blob/main/examples/reducing_hallucinations_ai_agents diff --git a/docs/src/examples/python_examples/build_from_scratch.md b/docs/src/examples/python_examples/build_from_scratch.md index 65e21af4..7019a810 100644 --- a/docs/src/examples/python_examples/build_from_scratch.md +++ b/docs/src/examples/python_examples/build_from_scratch.md @@ -1,10 +1,10 @@ # **Build from Scratch with LanceDB πŸ› οΈπŸš€** -Start building your GenAI applications from the ground up using LanceDB's efficient vector-based document retrieval capabilities! πŸ“‘ +Start building your GenAI applications from the ground up using **LanceDB's** efficient vector-based document retrieval capabilities! πŸ“‘ **Get Started in Minutes ⏱️** -These examples provide a solid foundation for building your own GenAI applications using LanceDB. Jump from idea to proof of concept quickly with applied examples. Get started and see what you can create! πŸ’» +These examples provide a solid foundation for building your own GenAI applications using LanceDB. Jump from idea to **proof of concept** quickly with applied examples. Get started and see what you can create! πŸ’» | **Build From Scratch** | **Description** | **Links** | |:-------------------------------------------|:-------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| diff --git a/docs/src/examples/python_examples/chatbot.md b/docs/src/examples/python_examples/chatbot.md index a16848a6..6d1e59cd 100644 --- a/docs/src/examples/python_examples/chatbot.md +++ b/docs/src/examples/python_examples/chatbot.md @@ -1,7 +1,7 @@ -**Chatbot Application with LanceDB πŸ€–** +**Chatbot applications with LanceDB πŸ€–** ==================================================================== - Create an innovative chatbot application that utilizes LanceDB for efficient vector-based response generation! 🌐✨ + Create innovative chatbot applications that utilizes LanceDB for efficient vector-based response generation! 🌐✨ **Introduction πŸ‘‹βœ¨** @@ -10,12 +10,12 @@ | **Chatbot** | **Description** | **Links** | |:----------------|:-----------------|:-----------| -| **Databricks DBRX Website Bot ⚑️** | Unlock magical conversations with the Hogwarts chatbot, powered by Open-source RAG, DBRX, LanceDB, LLama-index, and Hugging Face Embeddings, delivering enchanting user experiences and spellbinding interactions ✨ | [![GitHub](../../assets/github.svg)][databricks_github]
[![Python](../../assets/python.svg)][databricks_python] | -| **CLI SDK Manual Chatbot Locally πŸ’»** | CLI chatbot for SDK/hardware documents, powered by Local RAG, LLama3, Ollama, LanceDB, and Openhermes Embeddings, built with Phidata Assistant and Knowledge Base for instant technical support πŸ€– | [![GitHub](../../assets/github.svg)][clisdk_github]
[![Python](../../assets/python.svg)][clisdk_python] | -| **Youtube Transcript Search QA Bot πŸ“Ή** | Unlock the power of YouTube transcripts with a Q&A bot, leveraging natural language search and LanceDB for effortless data management and instant answers πŸ’¬ | [![GitHub](../../assets/github.svg)][youtube_github]
[![Open In Collab](../../assets/colab.svg)][youtube_colab]
[![Python](../../assets/python.svg)][youtube_python] | -| **Code Documentation Q&A Bot with LangChain πŸ€–** | Revolutionize code documentation with a Q&A bot, powered by LangChain and LanceDB, allowing effortless querying of documentation using natural language, demonstrated with Numpy 1.26 docs πŸ“š | [![GitHub](../../assets/github.svg)][docs_github]
[![Open In Collab](../../assets/colab.svg)][docs_colab]
[![Python](../../assets/python.svg)][docs_python] | -| **Context-aware Chatbot using Llama 2 & LanceDB πŸ€–** | Experience the future of conversational AI with a context-aware chatbot, powered by Llama 2, LanceDB, and LangChain, enabling intuitive and meaningful conversations with your data πŸ“šπŸ’¬ | [![GitHub](../../assets/github.svg)][aware_github]
[![Open In Collab](../../assets/colab.svg)][aware_colab]
[![Ghost](../../assets/ghost.svg)][aware_ghost] | -| **Chat with csv using Hybrid Search πŸ“Š** | Revolutionize data interaction with a chat application that harnesses LanceDB's hybrid search capabilities to converse with CSV and Excel files, enabling efficient and scalable data exploration and analysis πŸš€ | [![GitHub](../../assets/github.svg)][csv_github]
[![Open In Collab](../../assets/colab.svg)][csv_colab]
[![Ghost](../../assets/ghost.svg)][csv_ghost] | +| **Databricks DBRX Website Bot ⚑️** | Engage with the **Hogwarts chatbot**, that uses Open-source RAG with **DBRX**, **LanceDB** and **LLama-index with Hugging Face Embeddings**, to provide interactive and engaging user experiences. ✨ | [![GitHub](../../assets/github.svg)][databricks_github]
[![Python](../../assets/python.svg)][databricks_python] | +| **CLI SDK Manual Chatbot Locally πŸ’»** | CLI chatbot for SDK/hardware documents using **Local RAG** with **LLama3**, **Ollama**, **LanceDB**, and **Openhermes Embeddings**, built with **Phidata** Assistant and Knowledge Base πŸ€– | [![GitHub](../../assets/github.svg)][clisdk_github]
[![Python](../../assets/python.svg)][clisdk_python] | +| **Youtube Transcript Search QA Bot πŸ“Ή** | Search through **youtube transcripts** using natural language with a Q&A bot, leveraging **LanceDB** for effortless data storage and management πŸ’¬ | [![GitHub](../../assets/github.svg)][youtube_github]
[![Open In Collab](../../assets/colab.svg)][youtube_colab]
[![Python](../../assets/python.svg)][youtube_python] | +| **Code Documentation Q&A Bot with LangChain πŸ€–** | Query your own documentation easily using questions in natural language with a Q&A bot, powered by **LangChain** and **LanceDB**, demonstrated with **Numpy 1.26 docs** πŸ“š | [![GitHub](../../assets/github.svg)][docs_github]
[![Open In Collab](../../assets/colab.svg)][docs_colab]
[![Python](../../assets/python.svg)][docs_python] | +| **Context-aware Chatbot using Llama 2 & LanceDB πŸ€–** | Build **conversational AI** with a **context-aware chatbot**, powered by **Llama 2**, **LanceDB**, and **LangChain**, that enables intuitive and meaningful conversations with your data πŸ“šπŸ’¬ | [![GitHub](../../assets/github.svg)][aware_github]
[![Open In Collab](../../assets/colab.svg)][aware_colab]
[![Ghost](../../assets/ghost.svg)][aware_ghost] | +| **Chat with csv using Hybrid Search πŸ“Š** | **Chat** application that interacts with **CSV** and **Excel files** using **LanceDB’s** hybrid search capabilities, performing direct operations on large-scale columnar data efficiently πŸš€ | [![GitHub](../../assets/github.svg)][csv_github]
[![Open In Collab](../../assets/colab.svg)][csv_colab]
[![Ghost](../../assets/ghost.svg)][csv_ghost] | [databricks_github]: https://github.com/lancedb/vectordb-recipes/blob/main/examples/databricks_DBRX_website_bot diff --git a/docs/src/examples/python_examples/evaluations.md b/docs/src/examples/python_examples/evaluations.md index 9ee1a10a..18d0ccba 100644 --- a/docs/src/examples/python_examples/evaluations.md +++ b/docs/src/examples/python_examples/evaluations.md @@ -1,18 +1,16 @@ **Evaluation: Assessing Text Performance with Precision πŸ“ŠπŸ’‘** ==================================================================== -**Evaluation Fundamentals πŸ“Š** - Evaluation is a comprehensive tool designed to measure the performance of text-based inputs, enabling data-driven optimization and improvement πŸ“ˆ. **Text Evaluation 101 πŸ“š** -By leveraging cutting-edge technologies, this provides a robust framework for evaluating reference and candidate texts across various metrics πŸ“Š, ensuring high-quality text outputs that meet specific requirements and standards πŸ“. +Using robust framework for assessing reference and candidate texts across various metricsπŸ“Š, ensure that the text outputs are high-quality and meet specific requirements and standardsπŸ“. | **Evaluation** | **Description** | **Links** | | -------------- | --------------- | --------- | -| **Evaluating Prompts with Prompttools πŸ€–** | Compare, visualize & evaluate embedding functions (incl. OpenAI) across metrics like latency & custom evaluation πŸ“ˆπŸ“Š | [![Github](../../assets/github.svg)][prompttools_github]
[![Open In Collab](../../assets/colab.svg)][prompttools_colab] | -| **Evaluating RAG with RAGAs and GPT-4o πŸ“Š** | Evaluate RAG pipelines with cutting-edge metrics and tools, integrate with CI/CD for continuous performance checks, and generate responses with GPT-4o πŸ€–πŸ“ˆ | [![Github](../../assets/github.svg)][RAGAs_github]
[![Open In Collab](../../assets/colab.svg)][RAGAs_colab] | +| **Evaluating Prompts with Prompttools πŸ€–** | Compare, visualize & evaluate **embedding functions** (incl. OpenAI) across metrics like latency & custom evaluation πŸ“ˆπŸ“Š | [![Github](../../assets/github.svg)][prompttools_github]
[![Open In Collab](../../assets/colab.svg)][prompttools_colab] | +| **Evaluating RAG with RAGAs and GPT-4o πŸ“Š** | Evaluate **RAG pipelines** with cutting-edge metrics and tools, integrate with CI/CD for continuous performance checks, and generate responses with GPT-4o πŸ€–πŸ“ˆ | [![Github](../../assets/github.svg)][RAGAs_github]
[![Open In Collab](../../assets/colab.svg)][RAGAs_colab] | diff --git a/docs/src/examples/python_examples/multimodal.md b/docs/src/examples/python_examples/multimodal.md index 28ddce00..08c561c6 100644 --- a/docs/src/examples/python_examples/multimodal.md +++ b/docs/src/examples/python_examples/multimodal.md @@ -1,6 +1,6 @@ # **Multimodal Search with LanceDB πŸ€Ήβ€β™‚οΈπŸ”** -Experience the future of search with LanceDB's multimodal capabilities. Combine text and image queries to find the most relevant results in your corpus ! πŸ”“πŸ’‘ +Using LanceDB's multimodal capabilities, combine text and image queries to find the most relevant results in your corpus ! πŸ”“πŸ’‘ **Explore the Future of Search πŸš€** @@ -10,10 +10,10 @@ LanceDB supports multimodal search by indexing and querying vector representatio | **Multimodal** | **Description** | **Links** | |:----------------|:-----------------|:-----------| -| **Multimodal CLIP: DiffusionDB 🌐πŸ’₯** | Revolutionize search with Multimodal CLIP and DiffusionDB, combining text and image understanding for a new dimension of discovery! πŸ”“ | [![GitHub](../../assets/github.svg)][Clip_diffusionDB_github]
[![Open In Collab](../../assets/colab.svg)][Clip_diffusionDB_colab]
[![Python](../../assets/python.svg)][Clip_diffusionDB_python]
[![Ghost](../../assets/ghost.svg)][Clip_diffusionDB_ghost] | -| **Multimodal CLIP: Youtube Videos πŸ“ΉπŸ‘€** | Search Youtube videos using Multimodal CLIP, finding relevant content with ease and accuracy! 🎯 | [![Github](../../assets/github.svg)][Clip_youtube_github]
[![Open In Collab](../../assets/colab.svg)][Clip_youtube_colab]
[![Python](../../assets/python.svg)][Clip_youtube_python]
[![Ghost](../../assets/ghost.svg)][Clip_youtube_python] | -| **Multimodal Image + Text Search πŸ“ΈπŸ”** | Discover relevant documents and images with a single query, using LanceDB's multimodal search capabilities to bridge the gap between text and visuals! πŸŒ‰ | [![GitHub](../../assets/github.svg)](https://github.com/lancedb/vectordb-recipes/blob/main/examples/multimodal_search)
[![Open In Collab](../../assets/colab.svg)](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/multimodal_search/main.ipynb)
[![Python](../../assets/python.svg)](https://github.com/lancedb/vectordb-recipes/blob/main/examples/multimodal_search/main.py)
[![Ghost](../../assets/ghost.svg)](https://blog.lancedb.com/multi-modal-ai-made-easy-with-lancedb-clip-5aaf8801c939/) | -| **Cambrian-1: Vision-Centric Image Exploration πŸ”πŸ‘€** | Dive into vision-centric exploration of images with Cambrian-1, powered by LanceDB's multimodal search to uncover new insights! πŸ”Ž | [![Kaggle](https://img.shields.io/badge/Kaggle-035a7d?style=for-the-badge&logo=kaggle&logoColor=white)](https://www.kaggle.com/code/prasantdixit/cambrian-1-vision-centric-exploration-of-images/)
[![Ghost](../../assets/ghost.svg)](https://blog.lancedb.com/cambrian-1-vision-centric-exploration/) | +| **Multimodal CLIP: DiffusionDB 🌐πŸ’₯** | Multi-Modal Search with **CLIP** and **LanceDB** Using **DiffusionDB** Data for Combined Text and Image Understanding ! πŸ”“ | [![GitHub](../../assets/github.svg)][Clip_diffusionDB_github]
[![Open In Collab](../../assets/colab.svg)][Clip_diffusionDB_colab]
[![Python](../../assets/python.svg)][Clip_diffusionDB_python]
[![Ghost](../../assets/ghost.svg)][Clip_diffusionDB_ghost] | +| **Multimodal CLIP: Youtube Videos πŸ“ΉπŸ‘€** | Search **Youtube videos** using Multimodal CLIP, finding relevant content with ease and accuracy! 🎯 | [![Github](../../assets/github.svg)][Clip_youtube_github]
[![Open In Collab](../../assets/colab.svg)][Clip_youtube_colab]
[![Python](../../assets/python.svg)][Clip_youtube_python]
[![Ghost](../../assets/ghost.svg)][Clip_youtube_python] | +| **Multimodal Image + Text Search πŸ“ΈπŸ”** | Find **relevant documents** and **images** with a single query using **LanceDB's** multimodal search capabilities, to seamlessly integrate text and visuals ! πŸŒ‰ | [![GitHub](../../assets/github.svg)](https://github.com/lancedb/vectordb-recipes/blob/main/examples/multimodal_search)
[![Open In Collab](../../assets/colab.svg)](https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/multimodal_search/main.ipynb)
[![Python](../../assets/python.svg)](https://github.com/lancedb/vectordb-recipes/blob/main/examples/multimodal_search/main.py)
[![Ghost](../../assets/ghost.svg)](https://blog.lancedb.com/multi-modal-ai-made-easy-with-lancedb-clip-5aaf8801c939/) | +| **Cambrian-1: Vision-Centric Image Exploration πŸ”πŸ‘€** | Learn how **Cambrian-1** works, using an example of **Vision-Centric** exploration on images found through vector search ! Work on **Flickr-8k** dataset πŸ”Ž | [![Kaggle](https://img.shields.io/badge/Kaggle-035a7d?style=for-the-badge&logo=kaggle&logoColor=white)](https://www.kaggle.com/code/prasantdixit/cambrian-1-vision-centric-exploration-of-images/)
[![Ghost](../../assets/ghost.svg)](https://blog.lancedb.com/cambrian-1-vision-centric-exploration/) | [Clip_diffusionDB_github]: https://github.com/lancedb/vectordb-recipes/blob/main/examples/multimodal_clip_diffusiondb diff --git a/docs/src/examples/python_examples/rag.md b/docs/src/examples/python_examples/rag.md index 3d9f89fa..a6db3a68 100644 --- a/docs/src/examples/python_examples/rag.md +++ b/docs/src/examples/python_examples/rag.md @@ -1,5 +1,4 @@ - -**RAG: Revolutionize Information Retrieval with LanceDB πŸ”“πŸ§** +**RAG (Retrieval-Augmented Generation) with LanceDB πŸ”“πŸ§** ==================================================================== Build RAG (Retrieval-Augmented Generation) with LanceDB, a powerful solution for efficient vector-based information retrieval πŸ“Š. @@ -18,10 +17,10 @@ Build RAG (Retrieval-Augmented Generation) with LanceDB, a powerful solution fo | **Advanced RAG: Parent Document Retriever** πŸ“‘πŸ”— | Use **Parent Document & Bigger Chunk Retriever** to maintain context and relevance when generating related content. πŸŽ΅πŸ“„ | [![Github](../../assets/github.svg)][parent_doc_retriever_github]
[![Open In Collab](../../assets/colab.svg)][parent_doc_retriever_colab]
[![Ghost](../../assets/ghost.svg)][parent_doc_retriever_ghost] | | **Corrective RAG with Langgraph** πŸ”§πŸ“Š | Enhance RAG reliability with **Corrective RAG (CRAG)** by self-reflecting and fact-checking for accurate and trustworthy results. βœ…πŸ” |[![Github](../../assets/github.svg)][corrective_rag_github]
[![Open In Collab](../../assets/colab.svg)][corrective_rag_colab]
[![Ghost](../../assets/ghost.svg)][corrective_rag_ghost] | | **Contextual Compression with RAG** πŸ—œοΈπŸ§  | Apply **contextual compression techniques** to condense large documents while retaining essential information. πŸ“„πŸ—œοΈ | [![Github](../../assets/github.svg)][compression_rag_github]
[![Open In Collab](../../assets/colab.svg)][compression_rag_colab]
[![Ghost](../../assets/ghost.svg)][compression_rag_ghost] | -| **Improve RAG with FLARE** πŸ”₯| Enable users to ask questions directly to academic papers, focusing on ArXiv papers, with Forward-Looking Active REtrieval augmented generation.πŸš€πŸŒŸ | [![Github](../../assets/github.svg)][flare_github]
[![Open In Collab](../../assets/colab.svg)][flare_colab]
[![Ghost](../../assets/ghost.svg)][flare_ghost] | -| **Query Expansion and Reranker** πŸ”πŸ”„ | Enhance RAG with query expansion using Large Language Models and advanced **reranking methods** like Cross Encoders, ColBERT v2, and FlashRank for improved document retrieval precision and recall πŸ”πŸ“ˆ | [![Github](../../assets/github.svg)][query_github]
[![Open In Collab](../../assets/colab.svg)][query_colab] | -| **RAG Fusion** ⚑🌐 | Revolutionize search with RAG Fusion, utilizing the **RRF algorithm** to rerank documents based on user queries, and leveraging LanceDB and OPENAI Embeddings for efficient information retrieval ⚑🌐 | [![Github](../../assets/github.svg)][fusion_github]
[![Open In Collab](../../assets/colab.svg)][fusion_colab] | -| **Agentic RAG** πŸ€–πŸ“š | Unlock autonomous information retrieval with **Agentic RAG**, a framework of **intelligent agents** that collaborate to synthesize, summarize, and compare data across sources, enabling proactive and informed decision-making πŸ€–πŸ“š | [![Github](../../assets/github.svg)][agentic_github]
[![Open In Collab](../../assets/colab.svg)][agentic_colab] | +| **Improve RAG with FLARE** πŸ”₯| Enable users to ask questions directly to **academic papers**, focusing on **ArXiv papers**, with **F**orward-**L**ooking **A**ctive **RE**trieval augmented generation.πŸš€πŸŒŸ | [![Github](../../assets/github.svg)][flare_github]
[![Open In Collab](../../assets/colab.svg)][flare_colab]
[![Ghost](../../assets/ghost.svg)][flare_ghost] | +| **Query Expansion and Reranker** πŸ”πŸ”„ | Enhance RAG with query expansion using Large Language Models and advanced **reranking methods** like **Cross Encoders**, **ColBERT v2**, and **FlashRank** for improved document retrieval precision and recall πŸ”πŸ“ˆ | [![Github](../../assets/github.svg)][query_github]
[![Open In Collab](../../assets/colab.svg)][query_colab] | +| **RAG Fusion** ⚑🌐 | Build RAG Fusion, utilize the **RRF algorithm** to rerank documents based on user queries ! Use **LanceDB** as vector database to store and retrieve documents related to queries via **OPENAI Embeddings**⚑🌐 | [![Github](../../assets/github.svg)][fusion_github]
[![Open In Collab](../../assets/colab.svg)][fusion_colab] | +| **Agentic RAG** πŸ€–πŸ“š | Build autonomous information retrieval with **Agentic RAG**, a framework of **intelligent agents** that collaborate to synthesize, summarize, and compare data across sources, that enables proactive and informed decision-making πŸ€–πŸ“š | [![Github](../../assets/github.svg)][agentic_github]
[![Open In Collab](../../assets/colab.svg)][agentic_colab] | diff --git a/docs/src/examples/python_examples/recommendersystem.md b/docs/src/examples/python_examples/recommendersystem.md index ab7e4064..12ce7780 100644 --- a/docs/src/examples/python_examples/recommendersystem.md +++ b/docs/src/examples/python_examples/recommendersystem.md @@ -9,10 +9,10 @@ Deliver personalized experiences with Recommender Systems. 🎁 | **Recommender System** | **Description** | **Links** | | ---------------------- | --------------- | --------- | | **Movie Recommender System🎬** | 🀝 Use **collaborative filtering** to predict user preferences, assuming similar users will like similar movies, and leverage **Singular Value Decomposition** (SVD) from Numpy for precise matrix factorization and accurate recommendationsπŸ“Š | [![Github](../../assets/github.svg)][movie_github]
[![Open In Collab](../../assets/colab.svg)][movie_colab]
[![Python](../../assets/python.svg)][movie_python] | -| **πŸŽ₯ Movie Recommendation with Genres** | πŸ” Creates movie embeddings using Doc2Vec, capturing genre and characteristic nuances, and leverages VectorDB for efficient storage and querying, enabling accurate genre classification and personalized movie recommendations through similarity searchesπŸŽ₯ | [![Github](../../assets/github.svg)][genre_github]
[![Open In Collab](../../assets/colab.svg)][genre_colab]
[![Ghost](../../assets/ghost.svg)][genre_ghost] | +| **πŸŽ₯ Movie Recommendation with Genres** | πŸ” Creates movie embeddings using **Doc2Vec**, capturing genre and characteristic nuances, and leverages VectorDB for efficient storage and querying, enabling accurate genre classification and personalized movie recommendations through **similarity searches**πŸŽ₯ | [![Github](../../assets/github.svg)][genre_github]
[![Open In Collab](../../assets/colab.svg)][genre_colab]
[![Ghost](../../assets/ghost.svg)][genre_ghost] | | **πŸ›οΈ Product Recommender using Collaborative Filtering and LanceDB** | πŸ“ˆ Using **Collaborative Filtering** and **LanceDB** to analyze your past purchases, recommends products based on user's past purchases. Demonstrated with the Instacart dataset in our exampleπŸ›’ | [![Github](../../assets/github.svg)][product_github]
[![Open In Collab](../../assets/colab.svg)][product_colab]
[![Python](../../assets/python.svg)][product_python] | -| **πŸ” Arxiv Search with OpenCLIP and LanceDB** | πŸ’‘ Build a semantic search engine for Arxiv papers using LanceDB, and benchmarks its performance against traditional keyword-based search on Nomic's Atlas, to demonstrate the power of semantic search in finding relevant research papersπŸ“š | [![Github](../../assets/github.svg)][arxiv_github]
[![Open In Collab](../../assets/colab.svg)][arxiv_colab]
[![Python](../../assets/python.svg)][arxiv_python] | -| **Food Recommendation System🍴** | πŸ” Build a food recommendation system with LanceDB, featuring vector-based recommendations, full-text search, hybrid search, and reranking model integration for personalized and accurate food suggestionsπŸ‘Œ | [![Github](../../assets/github.svg)][food_github]
[![Open In Collab](../../assets/colab.svg)][food_colab] | +| **πŸ” Arxiv Search with OpenCLIP and LanceDB** | πŸ’‘ Build a semantic search engine for **Arxiv papers** using **LanceDB**, and benchmarks its performance against traditional keyword-based search on **Nomic's Atlas**, to demonstrate the power of semantic search in finding relevant research papersπŸ“š | [![Github](../../assets/github.svg)][arxiv_github]
[![Open In Collab](../../assets/colab.svg)][arxiv_colab]
[![Python](../../assets/python.svg)][arxiv_python] | +| **Food Recommendation System🍴** | πŸ” Build a food recommendation system with **LanceDB**, featuring vector-based recommendations, full-text search, hybrid search, and reranking model integration for personalized and accurate food suggestionsπŸ‘Œ | [![Github](../../assets/github.svg)][food_github]
[![Open In Collab](../../assets/colab.svg)][food_colab] | [movie_github]: https://github.com/lancedb/vectordb-recipes/blob/main/examples/movie-recommender [movie_colab]: https://colab.research.google.com/github/lancedb/vectordb-recipes/blob/main/examples/movie-recommender/main.ipynb diff --git a/docs/src/examples/python_examples/vector_search.md b/docs/src/examples/python_examples/vector_search.md index 7182eb09..8561f716 100644 --- a/docs/src/examples/python_examples/vector_search.md +++ b/docs/src/examples/python_examples/vector_search.md @@ -1,4 +1,4 @@ -**Vector Search: Unlock Efficient Document Retrieval πŸ”“πŸ‘€** +**Vector Search: Efficient Retrieval πŸ”“πŸ‘€** ==================================================================== Vector search with LanceDB, is a solution for efficient and accurate similarity searches in large datasets πŸ“Š. @@ -9,19 +9,19 @@ LanceDB implements vector search algorithms for efficient document retrieval and | **Vector Search** | **Description** | **Links** | |:-----------------|:---------------|:---------| -| **Inbuilt Hybrid Search πŸ”„** | Combine the power of traditional search algorithms with LanceDB's vector-based search for a robust and efficient search experience πŸ“Š | [![Github](../../assets/github.svg)][inbuilt_hybrid_search_github]
[![Open In Collab](../../assets/colab.svg)][inbuilt_hybrid_search_colab] | -| **Hybrid Search with BM25 and LanceDB πŸ’‘** | Synergizes BM25's keyword-focused precision (term frequency, document length normalization, bias-free retrieval) with LanceDB's semantic understanding (contextual analysis, query intent alignment) for nuanced search results in complex datasets πŸ“ˆ | [![Github](../../assets/github.svg)][BM25_github]
[![Open In Collab](../../assets/colab.svg)][BM25_colab]
[![Ghost](../../assets/ghost.svg)][BM25_ghost] | -| **NER-powered Semantic Search πŸ”Ž** | Unlock contextual understanding with Named Entity Recognition (NER) methods: Dictionary-Based, Rule-Based, and Deep Learning-Based, to accurately identify and extract entities, enabling precise semantic search results πŸ—‚οΈ | [![Github](../../assets/github.svg)][NER_github]
[![Open In Collab](../../assets/colab.svg)][NER_colab]
[![Ghost](../../assets/ghost.svg)][NER_ghost]| -| **Audio Similarity Search using Vector Embeddings 🎡** | Create vector embeddings of audio files to find similar audio content, enabling efficient audio similarity search and retrieval in LanceDB's vector store πŸ“» |[![Github](../../assets/github.svg)][audio_search_github]
[![Open In Collab](../../assets/colab.svg)][audio_search_colab]
[![Python](../../assets/python.svg)][audio_search_python]| -| **LanceDB Embeddings API: Multi-lingual Semantic Search 🌎** | Build a universal semantic search table with LanceDB's Embeddings API, supporting multiple languages (e.g., English, French) using cohere's multi-lingual model, for accurate cross-lingual search results πŸ“„ | [![Github](../../assets/github.svg)][mls_github]
[![Open In Collab](../../assets/colab.svg)][mls_colab]
[![Python](../../assets/python.svg)][mls_python] | -| **Facial Recognition: Face Embeddings πŸ€–** | Detect, crop, and embed faces using Facenet, then store and query face embeddings in LanceDB for efficient facial recognition and top-K matching results πŸ‘₯ | [![Github](../../assets/github.svg)][fr_github]
[![Open In Collab](../../assets/colab.svg)][fr_colab] | -| **Sentiment Analysis: Hotel Reviews 🏨** | Analyze customer sentiments towards the hotel industry using BERT models, storing sentiment labels, scores, and embeddings in LanceDB, enabling queries on customer opinions and potential areas for improvement πŸ’¬ | [![Github](../../assets/github.svg)][sentiment_analysis_github]
[![Open In Collab](../../assets/colab.svg)][sentiment_analysis_colab]
[![Ghost](../../assets/ghost.svg)][sentiment_analysis_ghost] | -| **Vector Arithmetic with LanceDB βš–οΈ** | Unlock powerful semantic search capabilities by performing vector arithmetic on embeddings, enabling complex relationships and nuances in data to be captured, and simplifying the process of retrieving semantically similar results πŸ“Š | [![Github](../../assets/github.svg)][arithmetic_github]
[![Open In Collab](../../assets/colab.svg)][arithmetic_colab]
[![Ghost](../../assets/ghost.svg)][arithmetic_ghost] | -| **Imagebind Demo πŸ–ΌοΈ** | Explore the multi-modal capabilities of Imagebind through a Gradio app, leveraging LanceDB API for seamless image search and retrieval experiences πŸ“Έ | [![Github](../../assets/github.svg)][imagebind_github]
[![Open in Spaces](../../assets/open_hf_space.svg)][imagebind_huggingface] | -| **Search Engine using SAM & CLIP πŸ”** | Build a search engine within an image using SAM and CLIP models, enabling object-level search and retrieval, with LanceDB indexing and search capabilities to find the closest match between image embeddings and user queries πŸ“Έ | [![Github](../../assets/github.svg)][swi_github]
[![Open In Collab](../../assets/colab.svg)][swi_colab]
[![Ghost](../../assets/ghost.svg)][swi_ghost] | -| **Zero Shot Object Localization and Detection with CLIP πŸ”Ž** | Perform object detection on images using OpenAI's CLIP, enabling zero-shot localization and detection of objects, with capabilities to split images into patches, parse with CLIP, and plot bounding boxes πŸ“Š | [![Github](../../assets/github.svg)][zsod_github]
[![Open In Collab](../../assets/colab.svg)][zsod_colab] | -| **Accelerate Vector Search with OpenVINO πŸš€** | Boost vector search applications using OpenVINO, achieving significant speedups with CLIP for text-to-image and image-to-image searching, through PyTorch model optimization, FP16 and INT8 format conversion, and quantization with OpenVINO NNCF πŸ“ˆ | [![Github](../../assets/github.svg)][openvino_github]
[![Open In Collab](../../assets/colab.svg)][openvino_colab]
[![Ghost](../../assets/ghost.svg)][openvino_ghost] | -| **Zero-Shot Image Classification with CLIP and LanceDB πŸ“Έ** | Achieve zero-shot image classification using CLIP and LanceDB, enabling models to classify images without prior training on specific use cases, unlocking flexible and adaptable image classification capabilities πŸ”“ | [![Github](../../assets/github.svg)][zsic_github]
[![Open In Collab](../../assets/colab.svg)][zsic_colab]
[![Ghost](../../assets/ghost.svg)][zsic_ghost] | +| **Inbuilt Hybrid Search πŸ”„** | Perform hybrid search in **LanceDB** by combining the results of semantic and full-text search via a reranking algorithm of your choice πŸ“Š | [![Github](../../assets/github.svg)][inbuilt_hybrid_search_github]
[![Open In Collab](../../assets/colab.svg)][inbuilt_hybrid_search_colab] | +| **Hybrid Search with BM25 and LanceDB πŸ’‘** | Use **Synergizes BM25's** keyword-focused precision (term frequency, document length normalization, bias-free retrieval) with **LanceDB's** semantic understanding (contextual analysis, query intent alignment) for nuanced search results in complex datasets πŸ“ˆ | [![Github](../../assets/github.svg)][BM25_github]
[![Open In Collab](../../assets/colab.svg)][BM25_colab]
[![Ghost](../../assets/ghost.svg)][BM25_ghost] | +| **NER-powered Semantic Search πŸ”Ž** | Extract and identify essential information from text with Named Entity Recognition **(NER)** methods: Dictionary-Based, Rule-Based, and Deep Learning-Based, to accurately extract and categorize entities, enabling precise semantic search results πŸ—‚οΈ | [![Github](../../assets/github.svg)][NER_github]
[![Open In Collab](../../assets/colab.svg)][NER_colab]
[![Ghost](../../assets/ghost.svg)][NER_ghost]| +| **Audio Similarity Search using Vector Embeddings 🎡** | Create vector **embeddings of audio files** to find similar audio content, enabling efficient audio similarity search and retrieval in **LanceDB's** vector store πŸ“» |[![Github](../../assets/github.svg)][audio_search_github]
[![Open In Collab](../../assets/colab.svg)][audio_search_colab]
[![Python](../../assets/python.svg)][audio_search_python]| +| **LanceDB Embeddings API: Multi-lingual Semantic Search 🌎** | Build a universal semantic search table with **LanceDB's Embeddings API**, supporting multiple languages (e.g., English, French) using **cohere's** multi-lingual model, for accurate cross-lingual search results πŸ“„ | [![Github](../../assets/github.svg)][mls_github]
[![Open In Collab](../../assets/colab.svg)][mls_colab]
[![Python](../../assets/python.svg)][mls_python] | +| **Facial Recognition: Face Embeddings πŸ€–** | Detect, crop, and embed faces using Facenet, then store and query face embeddings in **LanceDB** for efficient facial recognition and top-K matching results πŸ‘₯ | [![Github](../../assets/github.svg)][fr_github]
[![Open In Collab](../../assets/colab.svg)][fr_colab] | +| **Sentiment Analysis: Hotel Reviews 🏨** | Analyze customer sentiments towards the hotel industry using **BERT models**, storing sentiment labels, scores, and embeddings in **LanceDB**, enabling queries on customer opinions and potential areas for improvement πŸ’¬ | [![Github](../../assets/github.svg)][sentiment_analysis_github]
[![Open In Collab](../../assets/colab.svg)][sentiment_analysis_colab]
[![Ghost](../../assets/ghost.svg)][sentiment_analysis_ghost] | +| **Vector Arithmetic with LanceDB βš–οΈ** | Perform **vector arithmetic** on embeddings, enabling complex relationships and nuances in data to be captured, and simplifying the process of retrieving semantically similar results πŸ“Š | [![Github](../../assets/github.svg)][arithmetic_github]
[![Open In Collab](../../assets/colab.svg)][arithmetic_colab]
[![Ghost](../../assets/ghost.svg)][arithmetic_ghost] | +| **Imagebind Demo πŸ–ΌοΈ** | Explore the multi-modal capabilities of **Imagebind** through a Gradio app, use **LanceDB API** for seamless image search and retrieval experiences πŸ“Έ | [![Github](../../assets/github.svg)][imagebind_github]
[![Open in Spaces](../../assets/open_hf_space.svg)][imagebind_huggingface] | +| **Search Engine using SAM & CLIP πŸ”** | Build a search engine within an image using **SAM** and **CLIP** models, enabling object-level search and retrieval, with LanceDB indexing and search capabilities to find the closest match between image embeddings and user queries πŸ“Έ | [![Github](../../assets/github.svg)][swi_github]
[![Open In Collab](../../assets/colab.svg)][swi_colab]
[![Ghost](../../assets/ghost.svg)][swi_ghost] | +| **Zero Shot Object Localization and Detection with CLIP πŸ”Ž** | Perform object detection on images using **OpenAI's CLIP**, enabling zero-shot localization and detection of objects, with capabilities to split images into patches, parse with CLIP, and plot bounding boxes πŸ“Š | [![Github](../../assets/github.svg)][zsod_github]
[![Open In Collab](../../assets/colab.svg)][zsod_colab] | +| **Accelerate Vector Search with OpenVINO πŸš€** | Boost vector search applications using **OpenVINO**, achieving significant speedups with **CLIP** for text-to-image and image-to-image searching, through PyTorch model optimization, FP16 and INT8 format conversion, and quantization with **OpenVINO NNCF** πŸ“ˆ | [![Github](../../assets/github.svg)][openvino_github]
[![Open In Collab](../../assets/colab.svg)][openvino_colab]
[![Ghost](../../assets/ghost.svg)][openvino_ghost] | +| **Zero-Shot Image Classification with CLIP and LanceDB πŸ“Έ** | Achieve zero-shot image classification using **CLIP** and **LanceDB**, enabling models to classify images without prior training on specific use cases, unlocking flexible and adaptable image classification capabilities πŸ”“ | [![Github](../../assets/github.svg)][zsic_github]
[![Open In Collab](../../assets/colab.svg)][zsic_colab]
[![Ghost](../../assets/ghost.svg)][zsic_ghost] |