mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 15:22:30 +00:00
Jack Ye
b21fb2a4dc
The pages describing this code lived in lancedb/docs, a repository with ten stars that neither contributing guide mentioned. A user who hit a wrong page while reading the source had to find a separate, near-invisible repository to fix it. These 102 pages now sit beside the code, and a documentation fix is a pull request in the repository the reader already had open. `docs/web` is a complete Mintlify site: `cd docs/web && mint dev` serves it with no other checkout. Enterprise pages stay private and are merged in when the published site is assembled, so links into them do not resolve here — every open-source page does. `docs/web-tests` holds the tests the code examples are extracted from. They ran against a published wheel before; here they can run against this repository's own SDK, so an example that stops compiling fails the pull request that broke it rather than being noticed later. Both directories sit beside `docs/src` and are invisible to it: mkdocs reads `docs_dir: src`, so the SDK reference builds and deploys exactly as before.
39 lines
2.0 KiB
Plaintext
39 lines
2.0 KiB
Plaintext
---
|
|
title: Instructor
|
|
sidebarTitle: Instructor
|
|
---
|
|
|
|
import { PyEmbeddingInstructorUsage } from '/snippets/integrations.mdx';
|
|
|
|
[Instructor](https://instructor-embedding.github.io/) is an instruction-finetuned text embedding model that can generate text embeddings tailored to any task (e.g. classification, retrieval, clustering, text evaluation, etc.) and domains (e.g. science, finance, etc.) by simply providing the task instruction, without any finetuning.
|
|
|
|
If you want to calculate customized embeddings for specific sentences, you can follow the unified template to write instructions.
|
|
|
|
<Info>
|
|
Represent the `domain` `text_type` for `task_objective`:
|
|
|
|
- `domain` is optional, and it specifies the domain of the text, e.g. science, finance, medicine, etc.
|
|
- `text_type` is required, and it specifies the encoding unit, e.g. sentence, document, paragraph, etc.
|
|
- `task_objective` is optional, and it specifies the objective of embedding, e.g. retrieve a document, classify the sentence, etc.
|
|
</Info>
|
|
|
|
More information about the model can be found at the [source URL](https://github.com/xlang-ai/instructor-embedding).
|
|
|
|
| Argument | Type | Default | Description |
|
|
|---|---|---|---|
|
|
| `name` | `str` | "hkunlp/instructor-base" | The name of the model to use |
|
|
| `batch_size` | `int` | `32` | The batch size to use when generating embeddings |
|
|
| `device` | `str` | `"cpu"` | The device to use when generating embeddings |
|
|
| `show_progress_bar` | `bool` | `True` | Whether to show a progress bar when generating embeddings |
|
|
| `normalize_embeddings` | `bool` | `True` | Whether to normalize the embeddings |
|
|
| `quantize` | `bool` | `False` | Whether to quantize the model |
|
|
| `source_instruction` | `str` | `"represent the document for retrieval"` | The instruction for the source column |
|
|
| `query_instruction` | `str` | `"represent the document for retrieving the most similar documents"` | The instruction for the query |
|
|
|
|
|
|
|
|
<CodeGroup>
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyEmbeddingInstructorUsage}
|
|
</CodeBlock>
|
|
</CodeGroup> |