diff --git a/docs/src/python/python.md b/docs/src/python/python.md index dd60451cc..f15c51159 100644 --- a/docs/src/python/python.md +++ b/docs/src/python/python.md @@ -26,6 +26,18 @@ is also an [asynchronous API client](#connections-asynchronous). ::: lancedb.db.DBConnection +::: lancedb.Session + +## Namespaces (Synchronous) + +A namespace-backed connection resolves tables through a +[Lance namespace](https://lancedb.github.io/lance-namespace/) service instead of +listing a storage directory. + +::: lancedb.connect_namespace + +::: lancedb.namespace.LanceNamespaceDBConnection + ## Tables (Synchronous) ::: lancedb.table.Table @@ -34,8 +46,12 @@ is also an [asynchronous API client](#connections-asynchronous). ::: lancedb.table.FragmentSummaryStats +::: lancedb.table.TableStatistics + ::: lancedb.table.Tags +::: lancedb.table.Branches + ## Expressions Type-safe expression builder for filters and projections. Use these instead @@ -62,10 +78,41 @@ of raw SQL strings with [where][lancedb.query.LanceQueryBuilder.where] and ::: lancedb.query.LanceHybridQueryBuilder +::: lancedb.query.LanceEmptyQueryBuilder + +::: lancedb.query.LanceTakeQueryBuilder + +## Full text queries + +Structured full text queries can be passed to +[Table.search][lancedb.table.Table.search] or +[AsyncTable.search][lancedb.table.AsyncTable.search] in place of a query string, +and combined with [BooleanQuery][lancedb.query.BooleanQuery]. + +::: lancedb.query.FullTextQuery + +::: lancedb.query.MatchQuery + +::: lancedb.query.PhraseQuery + +::: lancedb.query.BoostQuery + +::: lancedb.query.MultiMatchQuery + +::: lancedb.query.BooleanQuery + +::: lancedb.query.FullTextOperator + +::: lancedb.query.Occur + ## Embeddings ::: lancedb.embeddings.registry.EmbeddingFunctionRegistry +::: lancedb.embeddings.registry.get_registry + +::: lancedb.embeddings.registry.register + ::: lancedb.embeddings.base.EmbeddingFunctionConfig ::: lancedb.embeddings.base.EmbeddingFunction @@ -78,6 +125,34 @@ of raw SQL strings with [where][lancedb.query.LanceQueryBuilder.where] and ::: lancedb.embeddings.open_clip.OpenClipEmbeddings +::: lancedb.embeddings.bedrock.BedRockText + +::: lancedb.embeddings.cohere.CohereEmbeddingFunction + +::: lancedb.embeddings.gemini_text.GeminiText + +::: lancedb.embeddings.gte.GteEmbeddings + +::: lancedb.embeddings.instructor.InstructorEmbeddingFunction + +::: lancedb.embeddings.jinaai.JinaEmbeddings + +::: lancedb.embeddings.ollama.OllamaEmbeddings + +::: lancedb.embeddings.transformers.TransformersEmbeddingFunction + +::: lancedb.embeddings.transformers.ColbertEmbeddings + +::: lancedb.embeddings.voyageai.VoyageAIEmbeddingFunction + +::: lancedb.embeddings.watsonx.WatsonxEmbeddings + +::: lancedb.embeddings.colpali.ColPaliEmbeddings + +::: lancedb.embeddings.imagebind.ImageBindEmbeddings + +::: lancedb.embeddings.siglip.SigLipEmbeddings + ## Remote configuration ::: lancedb.remote.ClientConfig @@ -86,6 +161,14 @@ of raw SQL strings with [where][lancedb.query.LanceQueryBuilder.where] and ::: lancedb.remote.RetryConfig +::: lancedb.remote.TlsConfig + +::: lancedb.remote.HeaderProvider + +::: lancedb.remote.OAuthConfig + +::: lancedb.remote.OAuthFlowType + ## Context ::: lancedb.context.contextualize @@ -124,12 +207,37 @@ tokens = list(lancedb.tokenize("acme makes searchable data", ::: lancedb.index.FTS +::: lancedb.tokenize + +::: lancedb.FtsToken + +## Blobs + +Blob columns store large binary values out of line so they can be read lazily +instead of being materialized with the rest of the row. + +::: lancedb.blob + +::: lancedb.BlobType + +::: lancedb._blob.BlobFile + options: + show_root_full_path: false + ## Utilities ::: lancedb.schema.vector ::: lancedb.merge.LanceMergeInsertBuilder +::: lancedb.otel.instrument_lancedb_metrics + +## Exceptions + +::: lancedb.exceptions.MissingValueError + +::: lancedb.exceptions.MissingColumnError + ## Integrations ## Pydantic @@ -138,10 +246,28 @@ tokens = list(lancedb.tokenize("acme makes searchable data", ::: lancedb.pydantic.vector +::: lancedb.pydantic.Vector + +::: lancedb.pydantic.MultiVector + ::: lancedb.pydantic.LanceModel +## PyTorch + +::: lancedb.streaming.StreamingDataset + +::: lancedb.permutation.permutation_builder + +::: lancedb.permutation.PermutationBuilder + +::: lancedb.permutation.Permutation + +::: lancedb.permutation.Transforms + ## Reranking +::: lancedb.rerankers.base.Reranker + ::: lancedb.rerankers.linear_combination.LinearCombinationReranker ::: lancedb.rerankers.cohere.CohereReranker @@ -152,6 +278,18 @@ tokens = list(lancedb.tokenize("acme makes searchable data", ::: lancedb.rerankers.openai.OpenaiReranker +::: lancedb.rerankers.jinaai.JinaReranker + +::: lancedb.rerankers.rrf.RRFReranker + +::: lancedb.rerankers.mrr.MRRReranker + +::: lancedb.rerankers.answerdotai.AnswerdotaiRerankers + +::: lancedb.rerankers.voyageai.VoyageAIReranker + +::: lancedb.rerankers.watsonx.WatsonxReranker + ## Connections (Asynchronous) Connections represent a connection to a LanceDb database and @@ -161,6 +299,12 @@ can be used to create, list, or open tables. ::: lancedb.db.AsyncConnection +## Namespaces (Asynchronous) + +::: lancedb.connect_namespace_async + +::: lancedb.namespace.AsyncLanceNamespaceDBConnection + ## Tables (Asynchronous) Table hold your actual data as a collection of records / rows. @@ -169,6 +313,8 @@ Table hold your actual data as a collection of records / rows. ::: lancedb.table.AsyncTags +::: lancedb.table.AsyncBranches + ## Indices (Asynchronous) Indices can be created on a table to speed up queries. This section @@ -196,6 +342,10 @@ lists the indices that LanceDb supports. ::: lancedb.index.HnswFlat +::: lancedb.index.Fm + +::: lancedb.index.IndexConfig + ::: lancedb.table.IndexStatistics ## Querying (Asynchronous) @@ -222,3 +372,7 @@ rows nearest to a query vector and can be created with the ::: lancedb.query.AsyncHybridQuery options: inherited_members: true + +::: lancedb.query.AsyncTakeQuery + options: + inherited_members: true diff --git a/python/python/lancedb/db.py b/python/python/lancedb/db.py index 0db9003d1..120e569c2 100644 --- a/python/python/lancedb/db.py +++ b/python/python/lancedb/db.py @@ -359,7 +359,7 @@ class DBConnection(EnforceOverrides): Data is converted to Arrow before being written to disk. For maximum control over how data is saved, either provide the PyArrow schema to - convert to or else provide a [PyArrow Table](pyarrow.Table) directly. + convert to or else provide a [PyArrow Table][pyarrow.Table] directly. >>> import pyarrow as pa >>> custom_schema = pa.schema([ @@ -1529,7 +1529,7 @@ class AsyncConnection(object): Data is converted to Arrow before being written to disk. For maximum control over how data is saved, either provide the PyArrow schema to - convert to or else provide a [PyArrow Table](pyarrow.Table) directly. + convert to or else provide a [PyArrow Table][pyarrow.Table] directly. >>> import pyarrow as pa >>> custom_schema = pa.schema([ diff --git a/python/python/lancedb/query.py b/python/python/lancedb/query.py index b85f50d7b..33f72edcf 100644 --- a/python/python/lancedb/query.py +++ b/python/python/lancedb/query.py @@ -664,8 +664,9 @@ class Query(pydantic.BaseModel): - A higher number makes search more accurate but also slower. - - See discussion in [Querying an ANN Index][querying-an-ann-index] for - tuning advice. + - See discussion in + [Querying an ANN Index](https://lancedb.com/docs/indexing/) + for tuning advice. Will be None if this is not a vector search. refine_factor : Optional[int] @@ -673,8 +674,9 @@ class Query(pydantic.BaseModel): - A higher number makes search more accurate but also slower. - - See discussion in [Querying an ANN Index][querying-an-ann-index] for - tuning advice. + - See discussion in + [Querying an ANN Index](https://lancedb.com/docs/indexing/) + for tuning advice. Will be None if this is not a vector search. lower_bound : Optional[float] @@ -1651,8 +1653,8 @@ class LanceVectorQueryBuilder(LanceQueryBuilder): Higher values will yield better recall (more likely to find vectors if they exist) at the expense of latency. - See discussion in [Querying an ANN Index][querying-an-ann-index] for - tuning advice. + See discussion in [Querying an ANN Index](https://lancedb.com/docs/indexing/) + for tuning advice. This method sets both the minimum and maximum number of probes to the same value. See `minimum_nprobes` and `maximum_nprobes` for more fine-grained @@ -1752,8 +1754,8 @@ class LanceVectorQueryBuilder(LanceQueryBuilder): As an example, a refine factor of 2 will sample 2x as many vectors as requested, re-ranks them, and returns the top half most relevant results. - See discussion in [Querying an ANN Index][querying-an-ann-index] for - tuning advice. + See discussion in [Querying an ANN Index](https://lancedb.com/docs/indexing/) + for tuning advice. Parameters ---------- diff --git a/python/python/lancedb/remote/table.py b/python/python/lancedb/remote/table.py index 01b90c019..628a98983 100644 --- a/python/python/lancedb/remote/table.py +++ b/python/python/lancedb/remote/table.py @@ -580,8 +580,9 @@ class RemoteTable(Table): progress: Optional[Union[bool, Callable, Any]] = None, write_parallelism: Optional[int] = None, ) -> AddResult: - """Add more data to the [Table](Table). It has the same API signature as - the OSS version. + """Add more data to the [Table][lancedb.table.Table]. + + It has the same API signature as the OSS version. Parameters ---------- diff --git a/python/python/lancedb/table.py b/python/python/lancedb/table.py index 032c6dce7..ca65b7a10 100644 --- a/python/python/lancedb/table.py +++ b/python/python/lancedb/table.py @@ -1211,7 +1211,7 @@ class Table(ABC): progress: Optional[Union[bool, Callable, Any]] = None, write_parallelism: Optional[int] = None, ) -> AddResult: - """Add more data to the [Table](Table). + """Add more data to the [Table][lancedb.table.Table]. Parameters ---------- @@ -1343,8 +1343,8 @@ class Table(ABC): fts_columns: Optional[Union[str, List[str]]] = None, ) -> LanceQueryBuilder: """Create a search query to find the nearest neighbors - of the given query vector. We currently support [vector search][search] - and [full-text search][experimental-full-text-search]. + of the given query vector. We currently support [vector search](https://lancedb.com/docs/search/vector-search/) + and [full-text search](https://lancedb.com/docs/search/full-text-search/). All query options are defined in [LanceQueryBuilder][lancedb.query.LanceQueryBuilder]. @@ -1778,7 +1778,7 @@ class Table(ABC): for faster reads. Arguments are passed onto Lance's - [compact_files][lance.dataset.DatasetOptimizer.compact_files]. + `lance.dataset.DatasetOptimizer.compact_files`. For most cases, the default should be fine. See Also @@ -3387,7 +3387,7 @@ class LanceTable(Table): fts_columns: Optional[Union[str, List[str]]] = None, ) -> LanceQueryBuilder: """Create a search query to find the nearest neighbors - of the given query vector. We currently support [vector search][search] + of the given query vector. We currently support [vector search](https://lancedb.com/docs/search/vector-search/) and [full-text search][search]. Examples @@ -4691,7 +4691,7 @@ class AsyncTable: Parameters ---------- **kwargs - Forwarded to [`lance.dataset`][lance.dataset]. + Forwarded to `lance.dataset`. Returns ------- @@ -5010,7 +5010,7 @@ class AsyncTable: progress: Optional[Union[bool, Callable, Any]] = None, write_parallelism: Optional[int] = None, ) -> AddResult: - """Add more data to the [Table](Table). + """Add more data to the [AsyncTable][lancedb.table.AsyncTable]. Parameters ---------- @@ -5212,8 +5212,8 @@ class AsyncTable: fts_columns: Optional[Union[str, List[str]]] = None, ) -> Union[AsyncHybridQuery, AsyncFTSQuery, AsyncVectorQuery]: """Create a search query to find the nearest neighbors - of the given query vector. We currently support [vector search][search] - and [full-text search][experimental-full-text-search]. + of the given query vector. We currently support [vector search](https://lancedb.com/docs/search/vector-search/) + and [full-text search](https://lancedb.com/docs/search/full-text-search/). All query options are defined in [AsyncQuery][lancedb.query.AsyncQuery].