rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-05-16 03:20:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore: convert all js doc test to use snippet. (#881)

Browse Source
This commit is contained in:
Lei Xu
2024-01-28 11:39:25 -08:00
committed by Weston Pace
parent d811b89de2
commit e7fdb931de
12 changed files with 246 additions and 232 deletions
Download Patch File Download Diff File Expand all files Collapse all files

98
docs/src/sql.md
View File

@@ -8,7 +8,7 @@ option that performs the filter prior to vector search. This can be useful to na
the search space on a very large dataset to reduce query latency.
<!-- Setup Code
```python
```python
import lancedb
import numpy as np
uri = "data/sample-lancedb"
@@ -21,7 +21,7 @@ tbl = db.create_table("my_vectors", data=data)
```
-->
<!-- Setup Code
```javascript
```javascript
const vectordb = require('vectordb')
const db = await vectordb.connect('data/sample-lancedb')
@@ -34,6 +34,7 @@ const tbl = await db.createTable('myVectors', data)
-->
=== "Python"
```py
result = (
tbl.search([0.5, 0.2])
@@ -44,12 +45,9 @@ const tbl = await db.createTable('myVectors', data)
```
=== "JavaScript"
```javascript
let result = await tbl.search(Array(1536).fill(0.5))
.limit(1)
.filter("id = 10")
.prefilter(true)
.execute()
--8<-- "src/sql_legacy.ts:search"
```
## SQL filters
@@ -60,14 +58,14 @@ It can be used during vector search, update, and deletion operations.
Currently, Lance supports a growing list of SQL expressions.
* ``>``, ``>=``, ``<``, ``<=``, ``=``
* ``AND``, ``OR``, ``NOT``
* ``IS NULL``, ``IS NOT NULL``
* ``IS TRUE``, ``IS NOT TRUE``, ``IS FALSE``, ``IS NOT FALSE``
* ``IN``
* ``LIKE``, ``NOT LIKE``
* ``CAST``
* ``regexp_match(column, pattern)``
- `>`, `>=`, `<`, `<=`, `=`
- `AND`, `OR`, `NOT`
- `IS NULL`, `IS NOT NULL`
- `IS TRUE`, `IS NOT TRUE`, `IS FALSE`, `IS NOT FALSE`
- `IN`
- `LIKE`, `NOT LIKE`
- `CAST`
- `regexp_match(column, pattern)`
For example, the following filter string is acceptable:
@@ -82,29 +80,27 @@ For example, the following filter string is acceptable:
=== "Javascript"
```javascript
await tbl.search(Array(1536).fill(0))
.where("(item IN ('item 0', 'item 2')) AND (id > 10)")
.execute()
--8<-- "src/sql_legacy.ts:vec_search"
```
If your column name contains special characters or is a [SQL Keyword](https://docs.rs/sqlparser/latest/sqlparser/keywords/index.html),
you can use backtick (`` ` ``) to escape it. For nested fields, each segment of the
path must be wrapped in backticks.
=== "SQL"
```sql
`CUBE` = 10 AND `column name with space` IS NOT NULL
AND `nested with space`.`inner with space` < 2
```
!!! warning
Field names containing periods (``.``) are not supported.
!!!warning "Field names containing periods (`.`) are not supported."
Literals for dates, timestamps, and decimals can be written by writing the string
value after the type name. For example
=== "SQL"
```sql
date_col = date '2021-01-01'
and timestamp_col = timestamp '2021-01-01 00:00:00'
@@ -114,49 +110,47 @@ value after the type name. For example
For timestamp columns, the precision can be specified as a number in the type
parameter. Microsecond precision (6) is the default.
| SQL | Time unit |
|------------------|--------------|
| ``timestamp(0)`` | Seconds |
| ``timestamp(3)`` | Milliseconds |
| ``timestamp(6)`` | Microseconds |
| ``timestamp(9)`` | Nanoseconds |
| SQL | Time unit |
| -------------- | ------------ |
| `timestamp(0)` | Seconds |
| `timestamp(3)` | Milliseconds |
| `timestamp(6)` | Microseconds |
| `timestamp(9)` | Nanoseconds |
LanceDB internally stores data in [Apache Arrow](https://arrow.apache.org/) format.
The mapping from SQL types to Arrow types is:
| SQL type | Arrow type |
|----------|------------|
| ``boolean`` | ``Boolean`` |
| ``tinyint`` / ``tinyint unsigned`` | ``Int8`` / ``UInt8`` |
| ``smallint`` / ``smallint unsigned`` | ``Int16`` / ``UInt16`` |
| ``int`` or ``integer`` / ``int unsigned`` or ``integer unsigned`` | ``Int32`` / ``UInt32`` |
| ``bigint`` / ``bigint unsigned`` | ``Int64`` / ``UInt64`` |
| ``float`` | ``Float32`` |
| ``double`` | ``Float64`` |
| ``decimal(precision, scale)`` | ``Decimal128`` |
| ``date`` | ``Date32`` |
| ``timestamp`` | ``Timestamp`` [^1] |
| ``string`` | ``Utf8`` |
| ``binary`` | ``Binary`` |
| SQL type | Arrow type |
| --------------------------------------------------------- | ------------------ |
| `boolean` | `Boolean` |
| `tinyint` / `tinyint unsigned` | `Int8` / `UInt8` |
| `smallint` / `smallint unsigned` | `Int16` / `UInt16` |
| `int` or `integer` / `int unsigned` or `integer unsigned` | `Int32` / `UInt32` |
| `bigint` / `bigint unsigned` | `Int64` / `UInt64` |
| `float` | `Float32` |
| `double` | `Float64` |
| `decimal(precision, scale)` | `Decimal128` |
| `date` | `Date32` |
| `timestamp` | `Timestamp` [^1] |
| `string` | `Utf8` |
| `binary` | `Binary` |
[^1]: See precision mapping in previous table.
## Filtering without Vector Search
You can also filter your data without search.
=== "Python"
```python
tbl.search().where("id = 10").limit(10).to_arrow()
```
```python
tbl.search().where("id = 10").limit(10).to_arrow()
```
=== "JavaScript"
```javascript
await tbl.where('id = 10').limit(10).execute()
```
!!! warning
If your table is large, this could potentially return a very large
amount of data. Please be sure to use a `limit` clause unless
you're sure you want to return the whole result set.
```javascript
--8<---- "src/sql_legacy.ts:sql_search"
```
!!!warning "If your table is large, this could potentially return a very large amount of data. Please be sure to use a `limit` clause unless you're sure you want to return the whole result set."