mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-23 20:22:37 +00:00
Aviral Saxena
d826b6115a
Addresses option 2 of #4440. tokenize() builds the FTS analyzer on the client. For jieba/* and lindera/* that needs model files under LANCE_LANGUAGE_MODEL_HOME, and when they’re missing you only got lance’s raw Invalid directory path: ... error, even though server-side search works. Now, if a jieba/* or lindera/* tokenizer can’t be built, the error keeps the original cause and adds that tokenize() runs on the client, which directory the files need to be in, and that for remote tables they must match the server’s. I also updated the tokenize docs (Rust, Python, TS, generated JS docs) to say a mismatch can’t be detected, so tokens may differ from the server’s index. Behavior when the files exist is unchanged. Detecting a mismatch needs a server-side tokenize endpoint (option 1), so I left it out. The hint isn’t limited to remote tables, since Table::as_native is slated for removal and the wording fits local tables too. Tests: Rust test with mocked jieba/default index_details and an empty model dir, one checking other tokenizer errors don’t get the hint, and Python sync/async tests.
LanceDB JavaScript SDK
A JavaScript library for LanceDB.
Installation
npm install @lancedb/lancedb
This will download the appropriate native library for your platform. We currently support:
- Linux (x86_64 and aarch64 on glibc and musl)
- MacOS (Intel and ARM/M1/M2)
- Windows (x86_64 and aarch64)
Usage
Basic Example
import * as lancedb from "@lancedb/lancedb";
const db = await lancedb.connect("data/sample-lancedb");
const table = await db.createTable("my_table", [
{ id: 1, vector: [0.1, 1.0], item: "foo", price: 10.0 },
{ id: 2, vector: [3.9, 0.5], item: "bar", price: 20.0 },
]);
const results = await table.vectorSearch([0.1, 0.3]).limit(20).toArray();
console.log(results);
The quickstart contains more complete examples.
Development
See CONTRIBUTING.md for information on how to contribute to LanceDB.