rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2025-12-26 06:39:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
a9897d9d85f5c958e680ec69bf16da746a3a6b7e
lancedb/docs/src/notebooks/hybrid_search.ipynb
Wyatt Alt 0b45ef93c0 docs: assorted copyedits (#1998) 
This includes a handful of minor edits I made while reading the docs. In
addition to a few spelling fixes,
* standardize on "rerank" over "re-rank" in prose
* terminate sentences with periods or colons as appropriate
* replace some usage of dashes with colons, such as in "Try it yourself
- <link>"

All changes are surface-level. No changes to semantics or structure.

---------

Co-authored-by: Will Jones <willjones127@gmail.com>
2025-01-06 15:04:48 -08:00

2001 lines
97 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"cells": [
{
"cell_type": "markdown",
"id": "0daef1cd-9130-46b8-8eb8-1b721860e239",
"metadata": {
"id": "0daef1cd-9130-46b8-8eb8-1b721860e239"
},
"source": [
"# Example - Airbnb financial data search\n",
"\n",
"<a href=\"https://colab.research.google.com/github/lancedb/lancedb/blob/main/docs/src/notebooks/hybrid_search.ipynb\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"></a>\n",
"\n",
"The code below is an example of hybrid search, a search algorithm that combines FTS and vector search in LanceDB.\n",
"\n",
"Let's get stared with an example. In this notebook we'll use Airbnb financial data documents to search for \"the specific reasons for higher operating costs\" in a particular year."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "819fa612",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "819fa612",
"outputId": "f5593c76-573f-4a04-d0ce-aac7b7ee1466"
},
"outputs": [],
"source": [
"# Setup\n",
"!pip install lancedb pandas langchain langchain_openai langchain-community pypdf openai cohere tiktoken sentence_transformers tantivy==0.20.1"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "b6864d97-7f85-4d9c-bf05-e9cf9db29e81",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "b6864d97-7f85-4d9c-bf05-e9cf9db29e81",
"outputId": "6c6dd78d-3213-4bd8-9e74-5faba902e546"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"··········\n"
]
}
],
"source": [
"import os\n",
"import getpass\n",
"\n",
"# Set your OpenAI API key\n",
"os.environ[\"OPENAI_API_KEY\"] = getpass.getpass()\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "cfce9804-cd1c-48c3-acd2-e74eb4e290c7",
"metadata": {
"id": "cfce9804-cd1c-48c3-acd2-e74eb4e290c7"
},
"outputs": [],
"source": [
"def pretty_print(docs):\n",
" for doc in docs:\n",
" print(doc + \"\\n\\n\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "efb22cec-5a06-46ac-91c3-53f9b9090109",
"metadata": {
"id": "efb22cec-5a06-46ac-91c3-53f9b9090109"
},
"outputs": [],
"source": [
"from langchain_community.document_loaders import PyPDFLoader\n",
"from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
"\n",
"# Load $ABNB's financial report. This may take 1-2 minutes since the PDF is large\n",
"sec_filing_pdf = \"https://d18rn0p25nwr6d.cloudfront.net/CIK-0001559720/8a9ebed0-815a-469a-87eb-1767d21d8cec.pdf\"\n",
"\n",
"# Create your PDF loader\n",
"loader = PyPDFLoader(sec_filing_pdf)\n",
"\n",
"# Load the PDF document\n",
"documents = loader.load()\n",
"\n",
"# Chunk the financial report\n",
"text_splitter = RecursiveCharacterTextSplitter(chunk_size=1024, chunk_overlap=0)\n",
"docs = text_splitter.split_documents(documents)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d3c5ce69-0f75-44cb-9e49-9be665fc156e",
"metadata": {
"id": "d3c5ce69-0f75-44cb-9e49-9be665fc156e"
},
"outputs": [],
"source": [
"from langchain_community.vectorstores import LanceDB\n",
"from langchain_openai import OpenAIEmbeddings\n",
"import lancedb\n",
"\n",
"\n",
"embedding_function = OpenAIEmbeddings()\n",
"\n",
"db = lancedb.connect(\"~/langchain\")\n",
"\n",
"# Load the document into LanceDB\n",
"db = LanceDB.from_documents(docs, embedding_function, connection=db)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4bd12fb8",
"metadata": {},
"outputs": [],
"source": [
"table = db._table\n",
"table.create_fts_index(\"text\")"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "d959a80f-d568-48f4-9d14-7367bcc1ce8d",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 293
},
"id": "d959a80f-d568-48f4-9d14-7367bcc1ce8d",
"outputId": "b02f837d-7fa6-4ec2-b283-e170f5f67637"
},
"outputs": [
{
"data": {
"application/vnd.google.colaboratory.intrinsic+json": {
"summary": "{\n \"name\": \"table\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"vector\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"ddcfa6b1-3de8-4933-a187-6aa7b7ae87b4\",\n \"47f5dd55-b3e7-4879-afba-5ca9eea7341b\",\n \"c391b1e1-6f66-41f2-82ff-18db5a218303\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Class A common stock, par value $0.0001 per share ABNB The Nasdaq Stock Market\\nSecurities registered pursuant to Section 12(g) of the Act:\\nNone______________\\nIndicate by check mark if the registrant is a well-known seasoned issuer, as defined in Rule 405 of the Securities Act. Yes \\u2612 No \\u2610 \\nIndicate by check mark if the registrant is not required to file reports pursuant to Section 13 or Section 15(d) of the Exchange Act. Yes \\u2610 No \\u2612 \\nIndicate by check mark whether the registrant (1) has filed all reports required to be filed by Section 13 or 15(d) of the Securities Exchange Act of 1934 during the preceding 12\\nmonths (or for such shorter period that the registrant was required to file such reports), and (2) has been subject to such filing requirements for the past 90 days. Yes \\u2612 No \\u2610 \\nIndicate by check mark whether the registrant has submitted electronically every Interactive Data File required to be submitted pursuant to Rule 405 of Regulation S-T (\\u00a7 232.405 of\",\n \"As of June 30, 2022, the aggregate market value of the Class A common stock held by non-affiliates of the registrant was approximately $35.1 billion based upon the closing price\\nreported for such date on the NASDAQ Global Select Market.\\nAs of February 3, 2023, 408,928,427 shares of the registrant's Class A common stock were outstanding 222,400,067 shares of the registrant's Class B common stock were\\noutstanding, no shares of the registrant\\u2019s Class C common stock were outstanding, and 9,200,000 shares of the registrant\\u2019s Class H common stock were outstanding.\\n______________\\nDOCUMENTS INCORPORATED BY REFERENCE\\nThe information required by Part III of this Report, to the extent not set forth herein, is incorporated herein by reference from the registrant\\u2019s definitive proxy statement relating to the\",\n \"this chapter) during the preceding 12 months (or for such shorter period that the registrant was required to submit such files). Yes \\u2612 No \\u2610 \\nIndicate by check mark whether the registrant is a large accelerated filer, an accelerated filer, a non-accelerated filer, a smaller reporting company, or an emerging growth company.\\nSee the definitions of \\u201clarge accelerated filer,\\u201d \\u201caccelerated filer,\\u201d \\u201csmaller reporting company\\u201d and \\u201cemerging growth company\\u201d in Rule 12b-2 of the Exchange Act.\\nLarge accelerated filer \\u2612 Accelerated filer\\u2610 \\nNon-accelerated filer \\u2610 Smaller reporting company\\u2610 \\nEmerging growth company\\u2610 \\nIf an emerging growth company, indicate by check mark if the registrant has elected not to use the extended transition period for complying with any new or revised financial\\naccounting standards provided pursuant to Section 13(a) of the Exchange Act. \\u2610\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metadata\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
"type": "dataframe"
},
"text/html": [
"\n",
" <div id=\"df-c61f9cf5-c221-4b4f-8a2c-6d02d2b66ffe\" class=\"colab-df-container\">\n",
" <div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>vector</th>\n",
" <th>id</th>\n",
" <th>text</th>\n",
" <th>metadata</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>[-0.0016961554, -0.03531899, 0.011809787, -0.0...</td>\n",
" <td>5c66d086-0fed-4270-a91b-c2b67b3ed052</td>\n",
" <td>Table of Contents\\nUNITED STATES\\nSECURITIES A...</td>\n",
" <td>{'page': 0, 'source': 'https://d18rn0p25nwr6d....</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>[-0.021446472, -0.021045355, 0.010823516, -0.0...</td>\n",
" <td>ddcfa6b1-3de8-4933-a187-6aa7b7ae87b4</td>\n",
" <td>Class A common stock, par value $0.0001 per sh...</td>\n",
" <td>{'page': 0, 'source': 'https://d18rn0p25nwr6d....</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>[-0.020018686, -0.014233166, -0.010991167, -0....</td>\n",
" <td>c391b1e1-6f66-41f2-82ff-18db5a218303</td>\n",
" <td>this chapter) during the preceding 12 months (...</td>\n",
" <td>{'page': 0, 'source': 'https://d18rn0p25nwr6d....</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>[-0.019061018, -0.0022632438, -0.011158161, -0...</td>\n",
" <td>3e896a62-8631-4a54-86bd-ee2f69f3b373</td>\n",
" <td>Indicate by check mark whether the registrant ...</td>\n",
" <td>{'page': 0, 'source': 'https://d18rn0p25nwr6d....</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>[-0.015733723, -0.012287037, -0.004055117, -0....</td>\n",
" <td>47f5dd55-b3e7-4879-afba-5ca9eea7341b</td>\n",
" <td>As of June 30, 2022, the aggregate market valu...</td>\n",
" <td>{'page': 1, 'source': 'https://d18rn0p25nwr6d....</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>\n",
" <div class=\"colab-df-buttons\">\n",
"\n",
" <div class=\"colab-df-container\">\n",
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c61f9cf5-c221-4b4f-8a2c-6d02d2b66ffe')\"\n",
" title=\"Convert this dataframe to an interactive table.\"\n",
" style=\"display:none;\">\n",
"\n",
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
" </svg>\n",
" </button>\n",
"\n",
" <style>\n",
" .colab-df-container {\n",
" display:flex;\n",
" gap: 12px;\n",
" }\n",
"\n",
" .colab-df-convert {\n",
" background-color: #E8F0FE;\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: #1967D2;\n",
" height: 32px;\n",
" padding: 0 0 0 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-convert:hover {\n",
" background-color: #E2EBFA;\n",
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: #174EA6;\n",
" }\n",
"\n",
" .colab-df-buttons div {\n",
" margin-bottom: 4px;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert {\n",
" background-color: #3B4455;\n",
" fill: #D2E3FC;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert:hover {\n",
" background-color: #434B5C;\n",
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
" fill: #FFFFFF;\n",
" }\n",
" </style>\n",
"\n",
" <script>\n",
" const buttonEl =\n",
" document.querySelector('#df-c61f9cf5-c221-4b4f-8a2c-6d02d2b66ffe button.colab-df-convert');\n",
" buttonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
"\n",
" async function convertToInteractive(key) {\n",
" const element = document.querySelector('#df-c61f9cf5-c221-4b4f-8a2c-6d02d2b66ffe');\n",
" const dataTable =\n",
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
" [key], {});\n",
" if (!dataTable) return;\n",
"\n",
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
" + ' to learn more about interactive tables.';\n",
" element.innerHTML = '';\n",
" dataTable['output_type'] = 'display_data';\n",
" await google.colab.output.renderOutput(dataTable, element);\n",
" const docLink = document.createElement('div');\n",
" docLink.innerHTML = docLinkHtml;\n",
" element.appendChild(docLink);\n",
" }\n",
" </script>\n",
" </div>\n",
"\n",
"\n",
"<div id=\"df-bd4a174f-e842-4ec2-9c02-05c752df5c4e\">\n",
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-bd4a174f-e842-4ec2-9c02-05c752df5c4e')\"\n",
" title=\"Suggest charts\"\n",
" style=\"display:none;\">\n",
"\n",
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
" width=\"24px\">\n",
" <g>\n",
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
" </g>\n",
"</svg>\n",
" </button>\n",
"\n",
"<style>\n",
" .colab-df-quickchart {\n",
" --bg-color: #E8F0FE;\n",
" --fill-color: #1967D2;\n",
" --hover-bg-color: #E2EBFA;\n",
" --hover-fill-color: #174EA6;\n",
" --disabled-fill-color: #AAA;\n",
" --disabled-bg-color: #DDD;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-quickchart {\n",
" --bg-color: #3B4455;\n",
" --fill-color: #D2E3FC;\n",
" --hover-bg-color: #434B5C;\n",
" --hover-fill-color: #FFFFFF;\n",
" --disabled-bg-color: #3B4455;\n",
" --disabled-fill-color: #666;\n",
" }\n",
"\n",
" .colab-df-quickchart {\n",
" background-color: var(--bg-color);\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: var(--fill-color);\n",
" height: 32px;\n",
" padding: 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-quickchart:hover {\n",
" background-color: var(--hover-bg-color);\n",
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: var(--button-hover-fill-color);\n",
" }\n",
"\n",
" .colab-df-quickchart-complete:disabled,\n",
" .colab-df-quickchart-complete:disabled:hover {\n",
" background-color: var(--disabled-bg-color);\n",
" fill: var(--disabled-fill-color);\n",
" box-shadow: none;\n",
" }\n",
"\n",
" .colab-df-spinner {\n",
" border: 2px solid var(--fill-color);\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" animation:\n",
" spin 1s steps(1) infinite;\n",
" }\n",
"\n",
" @keyframes spin {\n",
" 0% {\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" border-left-color: var(--fill-color);\n",
" }\n",
" 20% {\n",
" border-color: transparent;\n",
" border-left-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" }\n",
" 30% {\n",
" border-color: transparent;\n",
" border-left-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" border-right-color: var(--fill-color);\n",
" }\n",
" 40% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" }\n",
" 60% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" }\n",
" 80% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" border-bottom-color: var(--fill-color);\n",
" }\n",
" 90% {\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" }\n",
" }\n",
"</style>\n",
"\n",
" <script>\n",
" async function quickchart(key) {\n",
" const quickchartButtonEl =\n",
" document.querySelector('#' + key + ' button');\n",
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
" try {\n",
" const charts = await google.colab.kernel.invokeFunction(\n",
" 'suggestCharts', [key], {});\n",
" } catch (error) {\n",
" console.error('Error during call to suggestCharts:', error);\n",
" }\n",
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
" }\n",
" (() => {\n",
" let quickchartButtonEl =\n",
" document.querySelector('#df-bd4a174f-e842-4ec2-9c02-05c752df5c4e button');\n",
" quickchartButtonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
" })();\n",
" </script>\n",
"</div>\n",
"\n",
" </div>\n",
" </div>\n"
],
"text/plain": [
" vector \\\n",
"0 [-0.0016961554, -0.03531899, 0.011809787, -0.0... \n",
"1 [-0.021446472, -0.021045355, 0.010823516, -0.0... \n",
"2 [-0.020018686, -0.014233166, -0.010991167, -0.... \n",
"3 [-0.019061018, -0.0022632438, -0.011158161, -0... \n",
"4 [-0.015733723, -0.012287037, -0.004055117, -0.... \n",
"\n",
" id \\\n",
"0 5c66d086-0fed-4270-a91b-c2b67b3ed052 \n",
"1 ddcfa6b1-3de8-4933-a187-6aa7b7ae87b4 \n",
"2 c391b1e1-6f66-41f2-82ff-18db5a218303 \n",
"3 3e896a62-8631-4a54-86bd-ee2f69f3b373 \n",
"4 47f5dd55-b3e7-4879-afba-5ca9eea7341b \n",
"\n",
" text \\\n",
"0 Table of Contents\\nUNITED STATES\\nSECURITIES A... \n",
"1 Class A common stock, par value $0.0001 per sh... \n",
"2 this chapter) during the preceding 12 months (... \n",
"3 Indicate by check mark whether the registrant ... \n",
"4 As of June 30, 2022, the aggregate market valu... \n",
"\n",
" metadata \n",
"0 {'page': 0, 'source': 'https://d18rn0p25nwr6d.... \n",
"1 {'page': 0, 'source': 'https://d18rn0p25nwr6d.... \n",
"2 {'page': 0, 'source': 'https://d18rn0p25nwr6d.... \n",
"3 {'page': 0, 'source': 'https://d18rn0p25nwr6d.... \n",
"4 {'page': 1, 'source': 'https://d18rn0p25nwr6d.... "
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table.to_pandas().head()"
]
},
{
"cell_type": "markdown",
"id": "667f4e4a-6ff1-4f1c-ad57-4a2a8b036670",
"metadata": {
"id": "667f4e4a-6ff1-4f1c-ad57-4a2a8b036670"
},
"source": [
"## Vector Search\n",
"\n",
"Average latency: `3.48 ms ± 71.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)`"
]
},
{
"cell_type": "code",
"execution_count": 30,
"id": "8a5ab2de-6d75-4785-b838-ed6a825dfa6e",
"metadata": {
"id": "8a5ab2de-6d75-4785-b838-ed6a825dfa6e"
},
"outputs": [],
"source": [
"str_query = \"What are the specific factors contributing to Airbnb's increased operational expenses in the last fiscal year?\"\n",
"query = embedding_function.embed_query(str_query)\n",
"docs = table.search(query, query_type=\"vector\").limit(5).to_pandas()[\"text\"].to_list()"
]
},
{
"cell_type": "code",
"execution_count": 31,
"id": "5423d333-0f6d-4951-ab3f-6941ad30ba8a",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "5423d333-0f6d-4951-ab3f-6941ad30ba8a",
"outputId": "79557d98-85d1-4a18-db42-10d7adffb7c2"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatenedenforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood\n",
"associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict\n",
"home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust\n",
"and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful\n",
"\n",
"\n",
"Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising\n",
"spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider\n",
"expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to\n",
"the changes in the fair value of contingent consideration related to a 2019 acquisition.\n",
"General and Administrative\n",
"2021 2022 % Change\n",
"(in millions, except percentages)\n",
"General and administrative $ 836 $ 950 14 %\n",
"Percentage of revenue 14 % 11 %\n",
"General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3\n",
"\n",
"\n",
"Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially\n",
"adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:\n",
"• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic\n",
"downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other\n",
"climate change on seasonal destinations;\n",
"• political, social, or economic instability;\n",
"\n",
"\n",
"• Hosts failing to meet guests’ expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic;• increased competition and use of our competitors’ platforms and services;\n",
"• Hosts failing to provide differentiated, high-quality, and an adequate supply of stays or experiences at competitive prices;\n",
"• guests not receiving timely and adequate community support from us;\n",
"• our failure to provide new or enhanced offerings, tiers, or features that guests value;\n",
"• declines or inefficiencies in our marketing efforts;• negative associations with, or reduced awareness of, our brand;\n",
"• actual or perceived discrimination by Hosts in deciding whether to accept a requested reservation;\n",
"• negative perceptions of the trust and safety on our platform; and\n",
"• macroeconomic and other conditions outside of our control affecting travel and hospitality industries generally.\n",
"\n",
"\n",
"Table of Contents\n",
"Airbnb, Inc.\n",
"Consolidated Statements of Operations\n",
"(in millions, except per share amounts)\n",
"Year Ended December 31,\n",
"2020 2021 2022\n",
"Revenue $ 3,378 $ 5,992 $ 8,399 \n",
"Costs and expenses:\n",
"Cost of revenue 876 1,156 1,499 \n",
"Operations and support 878 847 1,041 \n",
"Product development 2,753 1,425 1,502 \n",
"Sales and marketing 1,175 1,186 1,516 \n",
"General and administrative 1,135 836 950 \n",
"Restructuring charges 151 113 89 \n",
"Total costs and expenses 6,968 5,563 6,597 \n",
"Income (loss) from operations (3,590) 429 1,802 \n",
"Interest income 27 13 186 \n",
"Interest expense (172) (438) (24)\n",
"Other income (expense), net (947) (304) 25 \n",
"Income (loss) before income taxes (4,682) (300) 1,989 \n",
"Provision for (benefit from) income taxes (97) 52 96 \n",
"Net income (loss) $ (4,585)$ (352)$ 1,893 \n",
"Net income (loss) per share attributable to Class A and Class B common stockholders:\n",
"Basic $ (16.12)$ (0.57)$ 2.97 \n",
"Diluted $ (16.12)$ (0.57)$ 2.79\n",
"\n",
"\n"
]
}
],
"source": [
"pretty_print(docs)"
]
},
{
"cell_type": "markdown",
"id": "8b0150fe-00dc-4aa0-9c8f-33cbf2ed5ac6",
"metadata": {
"id": "8b0150fe-00dc-4aa0-9c8f-33cbf2ed5ac6"
},
"source": [
"## Hybrid Search\n",
"LanceDB support hybrid search with custom Rerankers. Here's the summary of latency numbers of some of the Reranking methods available\n",
"![1_yWDh0Klw8Upsw1V54kkkdQ](https://github.com/AyushExel/assets/assets/15766192/a515fbf7-0553-437e-899e-67691eae3fef)\n",
"\n",
"Let us now perform hybrid search by combining vector and FTS search results. First, we'll cover the default Reranker.\n",
"\n",
"### Linear Combination Reranker\n",
"`LinearCombinationReranker(weight=0.7)` is used as the default reranker for reranking the hybrid search results if the reranker isn't specified explicitly.\n",
"The `weight` param controls the weightage provided to vector search score. The weight of `1-weight` is applied to FTS scores when reranking.\n",
"\n",
"Latency: `71 ms ± 25.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)`"
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "d2aa5893-30c4-4beb-9dae-a55665bd82c7",
"metadata": {
"id": "d2aa5893-30c4-4beb-9dae-a55665bd82c7"
},
"outputs": [],
"source": [
"docs = table.search(query_type=\"hybrid\").vector(query).text(str_query).limit(5).to_pandas()[\"text\"].to_list()"
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "8d6a99c3-92ef-4677-96bb-9b54a11a79fe",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "8d6a99c3-92ef-4677-96bb-9b54a11a79fe",
"outputId": "72f74e97-efb9-4bf2-c612-3dbbf33312bc"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatenedenforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood\n",
"associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict\n",
"home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust\n",
"and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful\n",
"\n",
"\n",
"“Initial Delivery Date”); provided that the Pricing Certificate for any fiscal year may be delivered on any date following the Initial DeliveryDate that is prior to the date that is 365 days following the last day of the preceding fiscal year, so long as such Pricing Certificate includes acertification that delivery of such Pricing Certificate on or before the Initial Delivery Date was not possible because (i) the informationrequired to calculate the KPI Metrics for such preceding fiscal year was not available at such time or (ii) the report of the KPI Metrics Auditor,if relevant, was not available at such time (the date of the Administrative Agent’s receipt thereof, each a “Pricing Certificate Date”). Upondelivery of a Pricing Certificate in respect of a fiscal year, (i) the Applicable Rate for the Loans incurred by the Borrower shall be increased ordecreased (or neither increased nor decreased), as applicable, pursuant to the Sustainability Margin Adjustment as set forth in the KPI MetricsCertificate\n",
"\n",
"\n",
"Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising\n",
"spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider\n",
"expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to\n",
"the changes in the fair value of contingent consideration related to a 2019 acquisition.\n",
"General and Administrative\n",
"2021 2022 % Change\n",
"(in millions, except percentages)\n",
"General and administrative $ 836 $ 950 14 %\n",
"Percentage of revenue 14 % 11 %\n",
"General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3\n",
"\n",
"\n",
"(c) If, for any fiscal year, either (i) no Pricing Certificate shall have been delivered for such fiscal year or (ii) the PricingCertificate delivered for such fiscal year shall fail to include the Diverse Supplier Spend Percentage or GHG Emissions Intensity for suchfiscal year, then the Sustainability Margin Adjustment will be positive 0.050% and/or the Sustainability Fee Adjustment will be positive0.010%, as applicable, in each case commencing on the last day such Pricing Certificate could have been delivered in accordance with theterms of clause (a) above (it being understood that, in the case of the foregoing clause (ii), the Sustainability Margin Adjustment or theSustainability Fee Adjustment will be determined in accordance with such Pricing Certificate to the extent the (A) Sustainability MarginAdjustment or the Sustainability Fee Adjustment is included in such Pricing Certificate and (B) the Administrative Agent has separatelyreceived the Diverse Supplier Spend Percentage and/or GHG Emissions\n",
"\n",
"\n",
"Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially\n",
"adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:\n",
"• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic\n",
"downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other\n",
"climate change on seasonal destinations;\n",
"• political, social, or economic instability;\n",
"\n",
"\n"
]
}
],
"source": [
"pretty_print(docs)"
]
},
{
"cell_type": "markdown",
"id": "c4d3e0f3-8d96-47f5-ad1d-514475f1ae55",
"metadata": {
"id": "c4d3e0f3-8d96-47f5-ad1d-514475f1ae55"
},
"source": [
"### Cohere Reranker\n",
"This uses Cohere's Reranking API to re-rank the results. It accepts the reranking model name as a parameter. By default it uses the english-v3 model but you can easily switch to a multi-lingual model.\n",
"\n",
"Latency: `605 ms ± 78.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)`"
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "ce2c43c7-1a96-4856-ad9b-28385164f187",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "ce2c43c7-1a96-4856-ad9b-28385164f187",
"outputId": "5316fdc4-8930-45aa-af1d-6f1faef4e97e"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"··········\n"
]
}
],
"source": [
"# Free API key\n",
"os.environ[\"COHERE_API_KEY\"] = getpass.getpass()"
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "4adbb3f1-4d21-427b-9bf0-3d7bebf68cf6",
"metadata": {
"id": "4adbb3f1-4d21-427b-9bf0-3d7bebf68cf6"
},
"outputs": [],
"source": [
"from lancedb.rerankers import CohereReranker\n",
"\n",
"reranker = CohereReranker()\n",
"docs = table.search(query_type=\"hybrid\").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()[\"text\"].to_list()"
]
},
{
"cell_type": "code",
"execution_count": 36,
"id": "a071b3e7-3b8b-42e4-a089-4d6c4094873f",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "a071b3e7-3b8b-42e4-a089-4d6c4094873f",
"outputId": "2d9066f3-8290-431d-ae08-0d17dad805f7"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the\n",
"above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition.\n",
"In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined\n",
"with sentiment of the workforce in China, and China’s policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that\n",
"our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create\n",
"\n",
"\n",
"Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising\n",
"spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider\n",
"expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to\n",
"the changes in the fair value of contingent consideration related to a 2019 acquisition.\n",
"General and Administrative\n",
"2021 2022 % Change\n",
"(in millions, except percentages)\n",
"General and administrative $ 836 $ 950 14 %\n",
"Percentage of revenue 14 % 11 %\n",
"General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3\n",
"\n",
"\n",
"• Hosts failing to meet guests’ expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic;• increased competition and use of our competitors’ platforms and services;\n",
"• Hosts failing to provide differentiated, high-quality, and an adequate supply of stays or experiences at competitive prices;\n",
"• guests not receiving timely and adequate community support from us;\n",
"• our failure to provide new or enhanced offerings, tiers, or features that guests value;\n",
"• declines or inefficiencies in our marketing efforts;• negative associations with, or reduced awareness of, our brand;\n",
"• actual or perceived discrimination by Hosts in deciding whether to accept a requested reservation;\n",
"• negative perceptions of the trust and safety on our platform; and\n",
"• macroeconomic and other conditions outside of our control affecting travel and hospitality industries generally.\n",
"\n",
"\n",
"Table of Contents\n",
"Airbnb, Inc.\n",
"Consolidated Statements of Operations\n",
"(in millions, except per share amounts)\n",
"Year Ended December 31,\n",
"2020 2021 2022\n",
"Revenue $ 3,378 $ 5,992 $ 8,399 \n",
"Costs and expenses:\n",
"Cost of revenue 876 1,156 1,499 \n",
"Operations and support 878 847 1,041 \n",
"Product development 2,753 1,425 1,502 \n",
"Sales and marketing 1,175 1,186 1,516 \n",
"General and administrative 1,135 836 950 \n",
"Restructuring charges 151 113 89 \n",
"Total costs and expenses 6,968 5,563 6,597 \n",
"Income (loss) from operations (3,590) 429 1,802 \n",
"Interest income 27 13 186 \n",
"Interest expense (172) (438) (24)\n",
"Other income (expense), net (947) (304) 25 \n",
"Income (loss) before income taxes (4,682) (300) 1,989 \n",
"Provision for (benefit from) income taxes (97) 52 96 \n",
"Net income (loss) $ (4,585)$ (352)$ 1,893 \n",
"Net income (loss) per share attributable to Class A and Class B common stockholders:\n",
"Basic $ (16.12)$ (0.57)$ 2.97 \n",
"Diluted $ (16.12)$ (0.57)$ 2.79\n",
"\n",
"\n",
"Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially\n",
"adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:\n",
"• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic\n",
"downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other\n",
"climate change on seasonal destinations;\n",
"• political, social, or economic instability;\n",
"\n",
"\n"
]
}
],
"source": [
"pretty_print(docs)"
]
},
{
"cell_type": "markdown",
"id": "6630f0c0-6070-4ea7-a191-99092e69ca05",
"metadata": {
"id": "6630f0c0-6070-4ea7-a191-99092e69ca05"
},
"source": [
"Relevance score is returned by Cohere API and is independent of individual FTS and vector search scores."
]
},
{
"cell_type": "code",
"execution_count": 38,
"id": "80dc61bb-929c-4fbb-b2cb-20c5d31bc65c",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 293
},
"id": "80dc61bb-929c-4fbb-b2cb-20c5d31bc65c",
"outputId": "d09dab34-7756-4c58-8731-5683b3ca7044"
},
"outputs": [
{
"data": {
"application/vnd.google.colaboratory.intrinsic+json": {
"summary": "{\n \"name\": \"table\",\n \"rows\": 5,\n \"fields\": [\n {\n \"column\": \"vector\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"id\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"a91b3506-39a2-4b19-8409-08333d83a1c6\",\n \"1694d5a5-7ece-40b8-8022-dc3fa9aaa05a\",\n \"fcc532b9-347b-4e36-8ae8-5a2a726bf574\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"text\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 5,\n \"samples\": [\n \"Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising\\nspend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider\\nexpenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to\\nthe changes in the fair value of contingent consideration related to a 2019 acquisition.\\nGeneral and Administrative\\n2021 2022 % Change\\n(in millions, except percentages)\\nGeneral and administrative $ 836 $ 950 14 %\\nPercentage of revenue 14 % 11 %\\nGeneral and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3\",\n \"Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially\\nadversely affected by a number of factors discussed elsewhere in these \\u201cRisk Factors,\\u201d including:\\n\\u2022 events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic\\ndownturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other\\nclimate change on seasonal destinations;\\n\\u2022 political, social, or economic instability;\",\n \"\\u2022 Hosts failing to meet guests\\u2019 expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic;\\u2022 increased competition and use of our competitors\\u2019 platforms and services;\\n\\u2022 Hosts failing to provide differentiated, high-quality, and an adequate supply of stays or experiences at competitive prices;\\n\\u2022 guests not receiving timely and adequate community support from us;\\n\\u2022 our failure to provide new or enhanced offerings, tiers, or features that guests value;\\n\\u2022 declines or inefficiencies in our marketing efforts;\\u2022 negative associations with, or reduced awareness of, our brand;\\n\\u2022 actual or perceived discrimination by Hosts in deciding whether to accept a requested reservation;\\n\\u2022 negative perceptions of the trust and safety on our platform; and\\n\\u2022 macroeconomic and other conditions outside of our control affecting travel and hospitality industries generally.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metadata\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"_relevance_score\",\n \"properties\": {\n \"dtype\": \"float32\",\n \"num_unique_values\": 5,\n \"samples\": [\n 0.9790357351303101,\n 0.5007786750793457,\n 0.961605966091156\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}",
"type": "dataframe"
},
"text/html": [
"\n",
" <div id=\"df-10faed38-ee75-4983-9cb3-5e04a6871893\" class=\"colab-df-container\">\n",
" <div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>vector</th>\n",
" <th>id</th>\n",
" <th>text</th>\n",
" <th>metadata</th>\n",
" <th>_relevance_score</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>[0.0034929817, -0.024774546, 0.012623285, -0.0...</td>\n",
" <td>18d4a926-99d9-447f-8b57-264d7a148bd7</td>\n",
" <td>Increased operating expenses, decreased revenu...</td>\n",
" <td>{'page': 18, 'source': 'https://d18rn0p25nwr6d...</td>\n",
" <td>0.985328</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>[-0.0042489874, -0.005382498, 0.007190078, -0....</td>\n",
" <td>a91b3506-39a2-4b19-8409-08333d83a1c6</td>\n",
" <td>Made Possible by Hosts, Strangers, AirCover, C...</td>\n",
" <td>{'page': 62, 'source': 'https://d18rn0p25nwr6d...</td>\n",
" <td>0.979036</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>[0.0076079983, -0.013340506, 0.018701892, -0.0...</td>\n",
" <td>fcc532b9-347b-4e36-8ae8-5a2a726bf574</td>\n",
" <td>• Hosts failing to meet guests’ expectations, ...</td>\n",
" <td>{'page': 11, 'source': 'https://d18rn0p25nwr6d...</td>\n",
" <td>0.961606</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>[-0.008694107, -0.01993283, 0.014201017, -0.02...</td>\n",
" <td>72b844e2-cc93-4495-bb67-c2c1a1fd6532</td>\n",
" <td>Table of Contents\\nAirbnb, Inc.\\nConsolidated ...</td>\n",
" <td>{'page': 72, 'source': 'https://d18rn0p25nwr6d...</td>\n",
" <td>0.696578</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>[0.005813433, -0.028278675, 0.018041687, -0.02...</td>\n",
" <td>1694d5a5-7ece-40b8-8022-dc3fa9aaa05a</td>\n",
" <td>Our success depends significantly on existing ...</td>\n",
" <td>{'page': 11, 'source': 'https://d18rn0p25nwr6d...</td>\n",
" <td>0.500779</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>\n",
" <div class=\"colab-df-buttons\">\n",
"\n",
" <div class=\"colab-df-container\">\n",
" <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-10faed38-ee75-4983-9cb3-5e04a6871893')\"\n",
" title=\"Convert this dataframe to an interactive table.\"\n",
" style=\"display:none;\">\n",
"\n",
" <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
" <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
" </svg>\n",
" </button>\n",
"\n",
" <style>\n",
" .colab-df-container {\n",
" display:flex;\n",
" gap: 12px;\n",
" }\n",
"\n",
" .colab-df-convert {\n",
" background-color: #E8F0FE;\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: #1967D2;\n",
" height: 32px;\n",
" padding: 0 0 0 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-convert:hover {\n",
" background-color: #E2EBFA;\n",
" box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: #174EA6;\n",
" }\n",
"\n",
" .colab-df-buttons div {\n",
" margin-bottom: 4px;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert {\n",
" background-color: #3B4455;\n",
" fill: #D2E3FC;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-convert:hover {\n",
" background-color: #434B5C;\n",
" box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
" filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
" fill: #FFFFFF;\n",
" }\n",
" </style>\n",
"\n",
" <script>\n",
" const buttonEl =\n",
" document.querySelector('#df-10faed38-ee75-4983-9cb3-5e04a6871893 button.colab-df-convert');\n",
" buttonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
"\n",
" async function convertToInteractive(key) {\n",
" const element = document.querySelector('#df-10faed38-ee75-4983-9cb3-5e04a6871893');\n",
" const dataTable =\n",
" await google.colab.kernel.invokeFunction('convertToInteractive',\n",
" [key], {});\n",
" if (!dataTable) return;\n",
"\n",
" const docLinkHtml = 'Like what you see? Visit the ' +\n",
" '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
" + ' to learn more about interactive tables.';\n",
" element.innerHTML = '';\n",
" dataTable['output_type'] = 'display_data';\n",
" await google.colab.output.renderOutput(dataTable, element);\n",
" const docLink = document.createElement('div');\n",
" docLink.innerHTML = docLinkHtml;\n",
" element.appendChild(docLink);\n",
" }\n",
" </script>\n",
" </div>\n",
"\n",
"\n",
"<div id=\"df-e8366691-e88e-4456-8672-836752a5a92e\">\n",
" <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e8366691-e88e-4456-8672-836752a5a92e')\"\n",
" title=\"Suggest charts\"\n",
" style=\"display:none;\">\n",
"\n",
"<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
" width=\"24px\">\n",
" <g>\n",
" <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
" </g>\n",
"</svg>\n",
" </button>\n",
"\n",
"<style>\n",
" .colab-df-quickchart {\n",
" --bg-color: #E8F0FE;\n",
" --fill-color: #1967D2;\n",
" --hover-bg-color: #E2EBFA;\n",
" --hover-fill-color: #174EA6;\n",
" --disabled-fill-color: #AAA;\n",
" --disabled-bg-color: #DDD;\n",
" }\n",
"\n",
" [theme=dark] .colab-df-quickchart {\n",
" --bg-color: #3B4455;\n",
" --fill-color: #D2E3FC;\n",
" --hover-bg-color: #434B5C;\n",
" --hover-fill-color: #FFFFFF;\n",
" --disabled-bg-color: #3B4455;\n",
" --disabled-fill-color: #666;\n",
" }\n",
"\n",
" .colab-df-quickchart {\n",
" background-color: var(--bg-color);\n",
" border: none;\n",
" border-radius: 50%;\n",
" cursor: pointer;\n",
" display: none;\n",
" fill: var(--fill-color);\n",
" height: 32px;\n",
" padding: 0;\n",
" width: 32px;\n",
" }\n",
"\n",
" .colab-df-quickchart:hover {\n",
" background-color: var(--hover-bg-color);\n",
" box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
" fill: var(--button-hover-fill-color);\n",
" }\n",
"\n",
" .colab-df-quickchart-complete:disabled,\n",
" .colab-df-quickchart-complete:disabled:hover {\n",
" background-color: var(--disabled-bg-color);\n",
" fill: var(--disabled-fill-color);\n",
" box-shadow: none;\n",
" }\n",
"\n",
" .colab-df-spinner {\n",
" border: 2px solid var(--fill-color);\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" animation:\n",
" spin 1s steps(1) infinite;\n",
" }\n",
"\n",
" @keyframes spin {\n",
" 0% {\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" border-left-color: var(--fill-color);\n",
" }\n",
" 20% {\n",
" border-color: transparent;\n",
" border-left-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" }\n",
" 30% {\n",
" border-color: transparent;\n",
" border-left-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" border-right-color: var(--fill-color);\n",
" }\n",
" 40% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" border-top-color: var(--fill-color);\n",
" }\n",
" 60% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" }\n",
" 80% {\n",
" border-color: transparent;\n",
" border-right-color: var(--fill-color);\n",
" border-bottom-color: var(--fill-color);\n",
" }\n",
" 90% {\n",
" border-color: transparent;\n",
" border-bottom-color: var(--fill-color);\n",
" }\n",
" }\n",
"</style>\n",
"\n",
" <script>\n",
" async function quickchart(key) {\n",
" const quickchartButtonEl =\n",
" document.querySelector('#' + key + ' button');\n",
" quickchartButtonEl.disabled = true; // To prevent multiple clicks.\n",
" quickchartButtonEl.classList.add('colab-df-spinner');\n",
" try {\n",
" const charts = await google.colab.kernel.invokeFunction(\n",
" 'suggestCharts', [key], {});\n",
" } catch (error) {\n",
" console.error('Error during call to suggestCharts:', error);\n",
" }\n",
" quickchartButtonEl.classList.remove('colab-df-spinner');\n",
" quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
" }\n",
" (() => {\n",
" let quickchartButtonEl =\n",
" document.querySelector('#df-e8366691-e88e-4456-8672-836752a5a92e button');\n",
" quickchartButtonEl.style.display =\n",
" google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
" })();\n",
" </script>\n",
"</div>\n",
"\n",
" </div>\n",
" </div>\n"
],
"text/plain": [
" vector \\\n",
"0 [0.0034929817, -0.024774546, 0.012623285, -0.0... \n",
"1 [-0.0042489874, -0.005382498, 0.007190078, -0.... \n",
"2 [0.0076079983, -0.013340506, 0.018701892, -0.0... \n",
"3 [-0.008694107, -0.01993283, 0.014201017, -0.02... \n",
"4 [0.005813433, -0.028278675, 0.018041687, -0.02... \n",
"\n",
" id \\\n",
"0 18d4a926-99d9-447f-8b57-264d7a148bd7 \n",
"1 a91b3506-39a2-4b19-8409-08333d83a1c6 \n",
"2 fcc532b9-347b-4e36-8ae8-5a2a726bf574 \n",
"3 72b844e2-cc93-4495-bb67-c2c1a1fd6532 \n",
"4 1694d5a5-7ece-40b8-8022-dc3fa9aaa05a \n",
"\n",
" text \\\n",
"0 Increased operating expenses, decreased revenu... \n",
"1 Made Possible by Hosts, Strangers, AirCover, C... \n",
"2 • Hosts failing to meet guests’ expectations, ... \n",
"3 Table of Contents\\nAirbnb, Inc.\\nConsolidated ... \n",
"4 Our success depends significantly on existing ... \n",
"\n",
" metadata _relevance_score \n",
"0 {'page': 18, 'source': 'https://d18rn0p25nwr6d... 0.985328 \n",
"1 {'page': 62, 'source': 'https://d18rn0p25nwr6d... 0.979036 \n",
"2 {'page': 11, 'source': 'https://d18rn0p25nwr6d... 0.961606 \n",
"3 {'page': 72, 'source': 'https://d18rn0p25nwr6d... 0.696578 \n",
"4 {'page': 11, 'source': 'https://d18rn0p25nwr6d... 0.500779 "
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"table.search(query_type=\"hybrid\").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()"
]
},
{
"cell_type": "markdown",
"id": "41147a46-7ef8-4266-9cec-08a992697de2",
"metadata": {
"id": "41147a46-7ef8-4266-9cec-08a992697de2"
},
"source": [
"### ColBERT Reranker\n",
"Colbert Reranker is powered by ColBERT model. It runs locally using the huggingface implementation.\n",
"\n",
"Latency - `950 ms ± 5.78 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)`\n",
"\n",
"Note: First query might be slow. It is recommended to reuse the `Reranker` objects as the models are cached. Subsequent runs will be faster on reusing the same reranker object"
]
},
{
"cell_type": "code",
"execution_count": 48,
"id": "zsV14JRXB0Xs",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "zsV14JRXB0Xs",
"outputId": "4900098d-0dd9-4a5b-9bec-14bad0a81f23"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting rerankers\n",
" Downloading rerankers-0.6.0-py3-none-any.whl.metadata (28 kB)\n",
"Requirement already satisfied: pydantic in /usr/local/lib/python3.10/dist-packages (from rerankers) (2.9.2)\n",
"Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from rerankers) (4.66.6)\n",
"Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from pydantic->rerankers) (0.7.0)\n",
"Requirement already satisfied: pydantic-core==2.23.4 in /usr/local/lib/python3.10/dist-packages (from pydantic->rerankers) (2.23.4)\n",
"Requirement already satisfied: typing-extensions>=4.6.1 in /usr/local/lib/python3.10/dist-packages (from pydantic->rerankers) (4.12.2)\n",
"Downloading rerankers-0.6.0-py3-none-any.whl (41 kB)\n",
"\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/41.1 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.1/41.1 kB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hInstalling collected packages: rerankers\n",
"Successfully installed rerankers-0.6.0\n"
]
}
],
"source": [
"!pip install rerankers"
]
},
{
"cell_type": "code",
"execution_count": 49,
"id": "91b06b43-c971-4177-b62f-f941bbbc2ef4",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 334,
"referenced_widgets": [
"d7770b21d69c4c45b6779a0a79a8d9c2",
"054a0b2d45914048a8ec508d9803c56c",
"add7e408b1734ea9a1d046ce4c39ef9e",
"47d9fc2c4dc44cc8a4d0f4a75f942c8a",
"a503998424394f34ba6f7c112f1c6efa",
"55c408272dd2489a99c831ff20ec9340",
"045197c90d5a4ccf8c0221a4020235da",
"3de12712877c483e8bb984d6eb004847",
"b26c3c7727204c22bd53e0111d221b86",
"fa2e6681eb8a41f1ba4cb22d46e1b7bc",
"c1b1423adfe140ec992dcd1a4ad36b1b",
"55a4200d05054bb295dc66d24acb10e8",
"3b396c309a054c1b8d16dc3ee3a97483",
"56ca1e56bc534e0db28f7e353e58b994",
"75a5a250d8dd408295e2897c22486091",
"fb8ffbf0ab8d4f43a09ee3a11ccdb950",
"1951dc337f584cafac599de1b73288ad",
"9ecf864eed024658bd5201d04d462564",
"7a532995568348b7b608844198848c9f",
"44b7024056de482a858b87ea8c2188f1",
"9acf476e0a7c4df59abd7ef2972775bf",
"c18d77fb1b8541eb937bcf4308992645",
"880662def20c41d1a419774b19ca259d",
"fc696ae381a54f6780f676981716e0f9",
"61f71d9f955c460283524a992d0492e1",
"7b5c08fc36f24bda9c44a89595bce448",
"d4c9ee37d58440b8a69207c90df95736",
"5f2fdccfc6f0445dbd08d92b389d96fc",
"7b5540ac6b4c4fd783e37765f4c51d77",
"5a4c3f55d98d4a3e97bc3dfef3a55dcf",
"44ba4ebbc71d4226a22ec4fc23bc8768",
"24d18127b4404cddb12b4a611bd20be8",
"ceb13b8e32774db183607d1424b9474f",
"b1b529b66911404792162c32b26245a1",
"63473a0152974d22aaa9e9396ef3e728",
"5defb1cd326f4b709d465e2b48a9edef",
"cb2942f98e354d788ae32d01fc59f5d5",
"b4a3413fddb84b62878a66241dbd4efb",
"6fb0e36f0283419f85a2b842b5a2b921",
"b5d6eb4b788946a8bf02c7ddb868eb32",
"7f138f0139294632bb50d446c164c967",
"cde7e23ade984cae9271d5db2a34a197",
"0851ea7f08724be4aa539fc91d66c642",
"a3d614e482994e28bad9644a6787b8db",
"d32c13984f204801860c6ac4563061bb",
"b49348ffe6ad4c5988dbc951f0bd22fd",
"3176ffcda7554b989b0cd23ad340bd92",
"6879ceca561f49e487cf05a3e7920f60",
"df82d77a80c14b9382c8674447447327",
"1cce1e346ff34714a489da3df078a703",
"e0f6ca6fe4e14037a3d513fbce236449",
"5d93708be5a94059b98de774aa5daa71",
"cd3763632f854fe4ada2952ca09245fb",
"69b7322fdcbe49f8a667707e3e0a3249",
"317ed2e5ae444ac5ba4fc07d360644b0",
"a80141d12cb94a2ca9564dd32af3b261",
"622bc2e5edee47c494265d35d0287d39",
"4299ffb4264c46b281ec1dda63a67e5c",
"77d0016687644feabf563a66eb40da85",
"47f0601a33c24f06b3bb10c6cd8dd9b3",
"dfdb3742e4174ab7b67dd83a5ae99711",
"a900bdccbb5141699ab7954a83a06d18",
"09ea0032eddc47bb84bc7d7f3cf7a2d6",
"7f9eacfcf78b42aa91fbc2641050c91c",
"acb2518611a24967a85afb0b60c6e9f5",
"3c728cf1d7f842529116636c1e1c929a"
]
},
"id": "91b06b43-c971-4177-b62f-f941bbbc2ef4",
"outputId": "ad72e836-4832-4d25-f64c-e439ef9f37d1"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Loading ColBERTRanker model colbert-ir/colbertv2.0 (this message can be suppressed by setting verbose=0)\n",
"No device set\n",
"Using device cpu\n",
"No dtype set\n",
"Using dtype torch.float32\n",
"Loading model colbert-ir/colbertv2.0, this might take a while...\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "d7770b21d69c4c45b6779a0a79a8d9c2",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"tokenizer_config.json: 0%| | 0.00/405 [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "55a4200d05054bb295dc66d24acb10e8",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"vocab.txt: 0%| | 0.00/232k [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "880662def20c41d1a419774b19ca259d",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"tokenizer.json: 0%| | 0.00/466k [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "b1b529b66911404792162c32b26245a1",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"special_tokens_map.json: 0%| | 0.00/112 [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "d32c13984f204801860c6ac4563061bb",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"config.json: 0%| | 0.00/743 [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "a80141d12cb94a2ca9564dd32af3b261",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"model.safetensors: 0%| | 0.00/438M [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Linear Dim set to: 128 for downcasting\n"
]
}
],
"source": [
"from lancedb.rerankers import ColbertReranker\n",
"\n",
"reranker = ColbertReranker()\n",
"docs = table.search(query_type=\"hybrid\").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()[\"text\"].to_list()"
]
},
{
"cell_type": "code",
"execution_count": 50,
"id": "e42c46bd-7cdd-4d31-9dbb-ddd1bdf979fa",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "e42c46bd-7cdd-4d31-9dbb-ddd1bdf979fa",
"outputId": "2b82a321-14d7-4cfa-e7e1-194888d5a5d6"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Table of Contents\n",
"Airbnb, Inc.\n",
"Consolidated Statements of Operations\n",
"(in millions, except per share amounts)\n",
"Year Ended December 31,\n",
"2020 2021 2022\n",
"Revenue $ 3,378 $ 5,992 $ 8,399 \n",
"Costs and expenses:\n",
"Cost of revenue 876 1,156 1,499 \n",
"Operations and support 878 847 1,041 \n",
"Product development 2,753 1,425 1,502 \n",
"Sales and marketing 1,175 1,186 1,516 \n",
"General and administrative 1,135 836 950 \n",
"Restructuring charges 151 113 89 \n",
"Total costs and expenses 6,968 5,563 6,597 \n",
"Income (loss) from operations (3,590) 429 1,802 \n",
"Interest income 27 13 186 \n",
"Interest expense (172) (438) (24)\n",
"Other income (expense), net (947) (304) 25 \n",
"Income (loss) before income taxes (4,682) (300) 1,989 \n",
"Provision for (benefit from) income taxes (97) 52 96 \n",
"Net income (loss) $ (4,585)$ (352)$ 1,893 \n",
"Net income (loss) per share attributable to Class A and Class B common stockholders:\n",
"Basic $ (16.12)$ (0.57)$ 2.97 \n",
"Diluted $ (16.12)$ (0.57)$ 2.79\n",
"\n",
"\n",
"In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatenedenforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood\n",
"associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict\n",
"home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust\n",
"and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful\n",
"\n",
"\n",
"Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the\n",
"above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition.\n",
"In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined\n",
"with sentiment of the workforce in China, and China’s policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that\n",
"our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create\n",
"\n",
"\n",
"Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising\n",
"spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider\n",
"expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to\n",
"the changes in the fair value of contingent consideration related to a 2019 acquisition.\n",
"General and Administrative\n",
"2021 2022 % Change\n",
"(in millions, except percentages)\n",
"General and administrative $ 836 $ 950 14 %\n",
"Percentage of revenue 14 % 11 %\n",
"General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3\n",
"\n",
"\n",
"Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially\n",
"adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:\n",
"• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic\n",
"downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other\n",
"climate change on seasonal destinations;\n",
"• political, social, or economic instability;\n",
"\n",
"\n"
]
}
],
"source": [
"pretty_print(docs)"
]
},
{
"cell_type": "markdown",
"id": "2ba9bc9a-29b0-4faa-b74d-a32af105ed45",
"metadata": {
"id": "2ba9bc9a-29b0-4faa-b74d-a32af105ed45"
},
"source": [
"### Cross Encoder Reranker\n",
"Uses cross encoder models are rerankers. Uses sentence transformer implementation locally\n",
"\n",
"Latency: `1.38 s ± 64.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)`"
]
},
{
"cell_type": "code",
"execution_count": 41,
"id": "4b9ea674-c8c6-498a-a3cf-9b7fa9cb7334",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 336,
"referenced_widgets": [
"0fc1ca4335e74d0a9d64290c111de78d",
"991d5513c344437f91f23b276b2c8382",
"77e08a2d2cbd4125987ce14321c425f2",
"afbe5b689e3e40f099a96b5b7b663cba",
"056af221639d411aa0a651eb48bbc12b",
"e5cc17b9ec4048a3b6a8efe7581993d6",
"4564cd9396c04b62b042c9ea55f98154",
"34a6065e6bc547879210db2ec55bd588",
"85a20895ff59444fa2cc758434afc8b2",
"9aa3992782a34b02879eaea1554a3721",
"33666ea759824b7b9aa243220b36bc13",
"9c11526d95054cd6a94be3d77ac110e7",
"6c4c0fa6eb734f8ab8f91144ce1898e5",
"bd39ea01d6a844fb9f763509192d9ba2",
"6d212c6ad52940ec8bdbd0561c49ce63",
"0a0d885f54d6436480cdf4f70a2d4ea1",
"885559b397304d7c97db71b8e5e7c568",
"9302da49d93646b7bd9ad11292ce986e",
"0103b977bf274e50b4b0b039dc643e6e",
"fa6e06107af24f7284c6f2b8936b28da",
"830f6e39bb5e4721a32d96a6c1feb9ba",
"9594c31e20be4f18b188c94495ffa5bd",
"ba6cd2d983ec4befb615b80e50833407",
"2ed3fe47a14f43b8bfc88f8736263feb",
"51995315b1bc445f9c650f72f7c68d5d",
"5eb2198b83cc441fa36ae3e7426821e7",
"86a1cf47d275427198a3fe641bc90972",
"ae13c273a9e64d93b6f2e51b1ade1e2c",
"48a3132cbec34e1eba59b8b24f002124",
"28fcf37a2da84d2a9a34f48df2541454",
"a6c4a14bd62c458790cb7311d6da7728",
"270ae007b83145bf8cd703d723264109",
"205812dd865e4c1b89bc9dcb90d87775",
"243dd02a7355446ea6f5f7b356220ecc",
"facff8557ac9419dafc8d2d7bd19af8d",
"3d6a89f74577485196cb4475b505978d",
"266c5ff52c8e4c0fb88dea4c5b216d91",
"fa2c1ae4370d4a9b9d28955141d641d5",
"070f1781a54344b2a6b2eb730a379139",
"c9362eefb4e2414385441fbb3cac376f",
"59b30a209f74488499b9452be64266f6",
"bf8f3ce222e24f91b5fee193042311b3",
"e0694db59d6d468dbef8627f4b6ee3e3",
"13070790677f45dda6f1205c3725699f",
"aa312b801b1a49a28a7ae8c14188e2bd",
"e192cd8f24f049babd866ece27e283e4",
"d9250bbb259d4f7fa501dc985e316f4d",
"05ff783a11914b0a8f3f7c636af96367",
"5e25713e2d6546daa1e6e9591c0a2bab",
"1a45a4da5d6340bb8ce5415a4904edf9",
"8a5d552d4721489abc294d379955022c",
"ff266892666b4f0083ce9fb3590798e8",
"2e04f04ae9584fbfbadaf12494bc7d35",
"a161dd1943ae46bcaac119c3a63e142d",
"7fd45ba46430486db69805c95b47d9b0",
"7d799d683f3d42af8b5ada98c0eea90c",
"dac9c266974d4bf0a2dddd5846038f0e",
"0418c934c62541cd8d8c9456b735604e",
"686a3aabd8e84b7da709cc100248ce1c",
"0e05bc02f73f4a0094abb5496cceeee1",
"81144371e6e0408b899ffd5112ca0e0a",
"1e6a2372b2374ee2ad5d02b32aaad78b",
"4caa2063a70641cf9228ec4447a9cea4",
"048bb9f1625f4313aa6f15cc29a714a8",
"77b60fc1763a4b7b98e8370855c861a5",
"73cc1833c95842f88d077721e6f2afe4"
]
},
"id": "4b9ea674-c8c6-498a-a3cf-9b7fa9cb7334",
"outputId": "112f2937-0e7d-4eb6-a876-20a0b1eaea86"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: \n",
"The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
"To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
"You will be able to reuse this secret in all of your notebooks.\n",
"Please note that authentication is recommended but still optional to access public models or datasets.\n",
" warnings.warn(\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "0fc1ca4335e74d0a9d64290c111de78d",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"config.json: 0%| | 0.00/612 [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "9c11526d95054cd6a94be3d77ac110e7",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"pytorch_model.bin: 0%| | 0.00/268M [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "ba6cd2d983ec4befb615b80e50833407",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"tokenizer_config.json: 0%| | 0.00/541 [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "243dd02a7355446ea6f5f7b356220ecc",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"vocab.txt: 0%| | 0.00/232k [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "aa312b801b1a49a28a7ae8c14188e2bd",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"special_tokens_map.json: 0%| | 0.00/112 [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "7d799d683f3d42af8b5ada98c0eea90c",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"model.safetensors: 0%| | 0.00/268M [00:00<?, ?B/s]"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from lancedb.rerankers import CrossEncoderReranker\n",
"\n",
"reranker=CrossEncoderReranker()\n",
"docs = table.search(query_type=\"hybrid\").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()[\"text\"].to_list()"
]
},
{
"cell_type": "code",
"execution_count": 42,
"id": "6fe32845-17f1-4977-9bd5-c18528b84656",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "6fe32845-17f1-4977-9bd5-c18528b84656",
"outputId": "f309ab49-d9c2-40c7-81a7-9dc5f7851839"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Table of Contents\n",
"Airbnb, Inc.\n",
"Consolidated Statements of Operations\n",
"(in millions, except per share amounts)\n",
"Year Ended December 31,\n",
"2020 2021 2022\n",
"Revenue $ 3,378 $ 5,992 $ 8,399 \n",
"Costs and expenses:\n",
"Cost of revenue 876 1,156 1,499 \n",
"Operations and support 878 847 1,041 \n",
"Product development 2,753 1,425 1,502 \n",
"Sales and marketing 1,175 1,186 1,516 \n",
"General and administrative 1,135 836 950 \n",
"Restructuring charges 151 113 89 \n",
"Total costs and expenses 6,968 5,563 6,597 \n",
"Income (loss) from operations (3,590) 429 1,802 \n",
"Interest income 27 13 186 \n",
"Interest expense (172) (438) (24)\n",
"Other income (expense), net (947) (304) 25 \n",
"Income (loss) before income taxes (4,682) (300) 1,989 \n",
"Provision for (benefit from) income taxes (97) 52 96 \n",
"Net income (loss) $ (4,585)$ (352)$ 1,893 \n",
"Net income (loss) per share attributable to Class A and Class B common stockholders:\n",
"Basic $ (16.12)$ (0.57)$ 2.97 \n",
"Diluted $ (16.12)$ (0.57)$ 2.79\n",
"\n",
"\n",
"Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising\n",
"spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider\n",
"expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to\n",
"the changes in the fair value of contingent consideration related to a 2019 acquisition.\n",
"General and Administrative\n",
"2021 2022 % Change\n",
"(in millions, except percentages)\n",
"General and administrative $ 836 $ 950 14 %\n",
"Percentage of revenue 14 % 11 %\n",
"General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3\n",
"\n",
"\n",
"Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the\n",
"above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition.\n",
"In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined\n",
"with sentiment of the workforce in China, and China’s policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that\n",
"our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create\n",
"\n",
"\n",
"In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatenedenforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood\n",
"associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict\n",
"home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust\n",
"and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful\n",
"\n",
"\n",
"Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially\n",
"adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:\n",
"• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic\n",
"downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other\n",
"climate change on seasonal destinations;\n",
"• political, social, or economic instability;\n",
"\n",
"\n"
]
}
],
"source": [
"pretty_print(docs)"
]
},
{
"cell_type": "markdown",
"id": "a32f41ea-e087-4e64-b9ec-f6224308fa6d",
"metadata": {
"id": "a32f41ea-e087-4e64-b9ec-f6224308fa6d"
},
"source": [
"### (Experimental) OpenAI Reranker\n",
"\n",
"This prompts a chat model to rerank results and is not a dedicated reranker model. This should be treated as experimental. You might exceed the token limit so set the search limits based on your token limit.\n",
"NOTE: It is recommended to use `gpt-4-turbo-preview` as older models might lead to bad behaviour\n",
"\n",
"Latency: `Can take 10s of seconds if using GPT-4 model`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "da78b250-9938-4e81-825f-c17b7a57e541",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 329
},
"id": "da78b250-9938-4e81-825f-c17b7a57e541",
"outputId": "f00802c2-d867-458a-efd4-d50f60a3c539"
},
"outputs": [],
"source": [
"from lancedb.rerankers import OpenaiReranker\n",
"\n",
"reranker=OpenaiReranker(model_name=\"gpt-4-turbo-preview\")\n",
"docs = table.search(query_type=\"hybrid\").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()[\"text\"].to_list()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "98e83f73-1ef3-485f-9871-9bd32937863f",
"metadata": {
"id": "98e83f73-1ef3-485f-9871-9bd32937863f"
},
"outputs": [],
"source": [
"pretty_print(docs)"
]
},
{
"cell_type": "markdown",
"id": "42dfdbc5-9006-4398-8465-03828ad48e49",
"metadata": {
"id": "42dfdbc5-9006-4398-8465-03828ad48e49"
},
"source": [
"## Use your custom Reranker\n",
"Hybrid search in LanceDB is designed to be very flexible. You can easily plug in your own Re-reranking logic. To do so, you simply need to implement the base Reranker class:"
]
},
{
"cell_type": "code",
"execution_count": 44,
"id": "e14503fe-5e9f-4d61-a96b-a5e95d501f61",
"metadata": {
"id": "e14503fe-5e9f-4d61-a96b-a5e95d501f61"
},
"outputs": [],
"source": [
"from lancedb.rerankers import Reranker\n",
"import pyarrow as pa\n",
"\n",
"class MyCustomReranker(Reranker):\n",
" def rerank_hybrid(self, query: str, vector_results: pa.Table, fts_results: pa.Table)-> pa.Table:\n",
" combined_results = self.merge(vector_results, fts_results) # Or custom merge algo\n",
" # Custom Reranking logic here\n",
"\n",
" return combined_results"
]
},
{
"cell_type": "markdown",
"id": "0606d4fb-96ef-4440-9363-f5461284d00c",
"metadata": {
"id": "0606d4fb-96ef-4440-9363-f5461284d00c"
},
"source": [
"### Custom Reranker based on CohereReranker\n",
"\n",
"For the sake of simplicity let's build a custom reranker that enhances the Cohere Reranker by accepting a filter query, and accepts other CohereReranker params as kwargs.\n",
"\n",
"For this toy example let's say we want to get rid of docs that represent a table of contents or appendix, as these are semantically close to representing costs but don't represent the specific reasons why operating costs were high."
]
},
{
"cell_type": "code",
"execution_count": 45,
"id": "dd1e8110-72c4-423c-90de-ce2b386742c1",
"metadata": {
"id": "dd1e8110-72c4-423c-90de-ce2b386742c1"
},
"outputs": [],
"source": [
"from typing import List, Union\n",
"import pandas as pd\n",
"from lancedb.rerankers import CohereReranker\n",
"\n",
"class MofidifiedCohereReranker(CohereReranker):\n",
" def __init__(self, filters: Union[str, List[str]], **kwargs):\n",
" super().__init__(**kwargs)\n",
" filters = filters if isinstance(filters, list) else [filters]\n",
" self.filters = filters\n",
"\n",
" def rerank_hybrid(self, query: str, vector_results: pa.Table, fts_results: pa.Table)-> pa.Table:\n",
" combined_result = super().rerank_hybrid(query, vector_results, fts_results)\n",
" df = combined_result.to_pandas()\n",
" for filter in self.filters:\n",
" df = df.query(\"not text.str.contains(@filter)\")\n",
"\n",
" return pa.Table.from_pandas(df)\n",
"\n",
"reranker = MofidifiedCohereReranker(filters=\"Table of Contents\")"
]
},
{
"cell_type": "code",
"execution_count": 46,
"id": "f4e6b496-e0c1-4944-8a6d-127f566812d3",
"metadata": {
"id": "f4e6b496-e0c1-4944-8a6d-127f566812d3"
},
"outputs": [],
"source": [
"docs = table.search(query_type=\"hybrid\").vector(query).text(str_query).limit(5).rerank(reranker).to_pandas()[\"text\"].to_list()"
]
},
{
"cell_type": "code",
"execution_count": 47,
"id": "5a29d0a2-793a-40a2-ac2d-2edda1102d6e",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "5a29d0a2-793a-40a2-ac2d-2edda1102d6e",
"outputId": "2aaf2369-0aa8-463d-9b81-da686f8adac5"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Increased operating expenses, decreased revenue, negative publicity, negative reaction from our Hosts and guests and other stakeholders, or other adverse impacts from any of the\n",
"above factors or other risks related to our international operations could materially adversely affect our brand, reputation, business, results of operations, and financial condition.\n",
"In addition, we will continue to incur significant expenses to operate our outbound business in China, and we may never achieve profitability in that market. These factors, combined\n",
"with sentiment of the workforce in China, and China’s policy towards foreign direct investment may particularly impact our operations in China. In addition, we need to ensure that\n",
"our business practices in China are compliant with local laws and regulations, which may be interpreted and enforced in ways that are different from our interpretation, and/or create\n",
"\n",
"\n",
"Made Possible by Hosts, Strangers, AirCover, Categories, and OMG marketing campaigns and launches, a $67.9 million increase in our search engine marketing and advertising\n",
"spend, a $25.1 million increase in payroll-related expenses due to growth in headcount and increase in compensation costs, a $22.0 million increase in third-party service provider\n",
"expenses, and a $11.1 million increase in coupon expense in line with increase in revenue and launch of AirCover for guests, partially offset by a decrease of $22.9 million related to\n",
"the changes in the fair value of contingent consideration related to a 2019 acquisition.\n",
"General and Administrative\n",
"2021 2022 % Change\n",
"(in millions, except percentages)\n",
"General and administrative $ 836 $ 950 14 %\n",
"Percentage of revenue 14 % 11 %\n",
"General and administrative expense increased $114.0 million, or 14%, in 2022 compared to 2021, primarily due to an increase in other business and operational taxes of $41.3\n",
"\n",
"\n",
"• Hosts failing to meet guests’ expectations, including increased expectations for cleanliness in light of the COVID-19 pandemic;• increased competition and use of our competitors’ platforms and services;\n",
"• Hosts failing to provide differentiated, high-quality, and an adequate supply of stays or experiences at competitive prices;\n",
"• guests not receiving timely and adequate community support from us;\n",
"• our failure to provide new or enhanced offerings, tiers, or features that guests value;\n",
"• declines or inefficiencies in our marketing efforts;• negative associations with, or reduced awareness of, our brand;\n",
"• actual or perceived discrimination by Hosts in deciding whether to accept a requested reservation;\n",
"• negative perceptions of the trust and safety on our platform; and\n",
"• macroeconomic and other conditions outside of our control affecting travel and hospitality industries generally.\n",
"\n",
"\n",
"Our success depends significantly on existing guests continuing to book and attracting new guests to book on our platform. Our ability to attract and retain guests could be materially\n",
"adversely affected by a number of factors discussed elsewhere in these “Risk Factors,” including:\n",
"• events beyond our control such as the ongoing COVID-19 pandemic, other pandemics and health concerns, restrictions on travel, immigration, trade disputes, economic\n",
"downturns, and the impact of climate change on travel including the availability of preferred destinations and the increase in the frequency and severity of weather-relatedevents, including fires, floods, droughts, extreme temperatures and ambient temperature increases, severe weather and other natural disasters, and the impact of other\n",
"climate change on seasonal destinations;\n",
"• political, social, or economic instability;\n",
"\n",
"\n",
"In addition, the number of listings on Airbnb may decline as a result of a number of other factors affecting Hosts, including: the COVID-19 pandemic; enforcement or threatenedenforcement of laws and regulations, including short-term occupancy and tax laws; private groups, such as homeowners, landlords, and condominium and neighborhood\n",
"associations, adopting and enforcing contracts that prohibit or restrict home sharing; leases, mortgages, and other agreements, or regulations that purport to ban or otherwise restrict\n",
"home sharing; Hosts opting for long-term rentals on other third-party platforms as an alternative to listing on our platform; economic, social, and political factors; perceptions of trust\n",
"and safety on and off our platform; negative experiences with guests, including guests who damage Host property, throw unauthorized parties, or engage in violent and unlawful\n",
"\n",
"\n"
]
}
],
"source": [
"pretty_print(docs)"
]
},
{
"cell_type": "markdown",
"id": "b3b5464a-7252-4eab-aaac-9b0eae37496f",
"metadata": {
"id": "b3b5464a-7252-4eab-aaac-9b0eae37496f"
},
"source": [
"As you can see, the document containing the table of contents no longer shows up."
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Reference in New Issue View Git Blame Copy Permalink