fix: improve error handling and embedding logic in arrow.ts (#2433)

- Enhanced error messages for schema inference failures to suggest
providing an explicit schema.
- Updated embedding application logic to check for existing destination
columns, allowing for filling embeddings in columns that are all null.
- Added comments for clarity on handling existing columns during
embedding application.

Fixes https://github.com/lancedb/lancedb/issues/2183

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

- **Bug Fixes**
  - Improved error messages for schema inference to enhance readability.
- Prevented redundant embedding application by skipping columns that
already contain data, avoiding unnecessary errors and computations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
satya-nutella
2025-06-18 12:45:11 -07:00
committed by GitHub
parent b77314168d
commit 9645fe52c2
2 changed files with 31 additions and 11 deletions
+4 -4
View File
@@ -592,14 +592,14 @@ describe.each([arrow15, arrow16, arrow17, arrow18])(
).rejects.toThrow("column vector was missing");
});
it("will provide a nice error if run twice", async function () {
it("will skip embedding application if already applied", async function () {
const records = sampleRecords();
const table = await convertToTable(records, dummyEmbeddingConfig);
// fromTableToBuffer will try and apply the embeddings again
await expect(
fromTableToBuffer(table, dummyEmbeddingConfig),
).rejects.toThrow("already existed");
// but should skip since the column already has non-null values
const result = await fromTableToBuffer(table, dummyEmbeddingConfig);
expect(result.byteLength).toBeGreaterThan(0);
});
});