rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-01-09 13:22:58 +00:00
Code Issues Packages Projects Releases Wiki Activity

Exponential standoff retry support for handling rate limited embedding functions (#614)

Browse Source 
Users ingesting data using rate limited apis don't need to manually make
the process sleep for counter rate limits
resolves #579
This commit is contained in:
Ayush Chaurasia
2023-11-02 19:20:10 +05:30
committed by GitHub
parent 682e95fa83
commit 1589499f89
6 changed files with 134 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
python/lancedb/table.py
View File

@@ -86,7 +86,9 @@ def _append_vector_col(data: pa.Table, metadata: dict, schema: Optional[pa.Schem
for vector_column, conf in functions.items():
func = conf.function
if vector_column not in data.column_names:
col_data = func.compute_source_embeddings(data[conf.source_column])
col_data = func.compute_source_embeddings_with_retry(
data[conf.source_column]
)
if schema is not None:
dtype = schema.field(vector_column).type
else: