mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 15:22:30 +00:00
docs: consolidate pages that were split for no reason
Ten merges and renames, all in one direction: fewer pages, each covering a whole subject. Table absorbs schema evolution and versioning. Vector search absorbs multi-vector. Explain absorbs explain analyze. Reranking was seven pages -- an overview and six rerankers of a few dozen lines each -- and is now one. The vector index page absorbs quantization, which was only ever a parameter of it. Indexing moves to Manage, where the other catalog objects are. The operational half stays in Operate as one Indexing page, absorbing GPU indexing -- which was filed in sophon despite importing lancedb's snippets and being mostly open-source content with one Enterprise-badged section. Performance was a single page aggregating tips for four unrelated operations, which is hard to keep current. Each section now sits with the operation it is about: ingestion tips on Insert, index tips on Indexing, compaction tips on Compaction, query tips on Search. Data Types leaves Concepts. The type reference belongs in the SQL reference, which already described the same types from the query side; Table now carries a short summary that points there. The multimodal material it had absorbed is its own page again, since storing blobs is a task rather than a type definition. Query becomes Semantic Search, Branches becomes Branch, and the Enterprise Security page becomes Compliance. Redirects cover every moved URL, and 30 files were repointed at the merged anchors.
This commit is contained in:
@@ -13,9 +13,32 @@ DataFusion the dataset's Arrow schema directly, so a column is the same object
|
||||
to the query engine as it is to the format.
|
||||
|
||||
The consequence is that SQL type names here are **spellings of Arrow types**,
|
||||
not a system that maps onto them. [Data types](/basics/data-types) documents the
|
||||
not a system that maps onto them. [Data types](/sql/data-types) documents the
|
||||
storage side; this page covers what changes when you reach a column from SQL.
|
||||
|
||||
## The Arrow types a column can be {#supported}
|
||||
|
||||
| Category | Types |
|
||||
| --- | --- |
|
||||
| Null | `null` |
|
||||
| Boolean | `bool` |
|
||||
| Integer | `int8`, `int16`, `int32`, `int64` and their `uint` counterparts |
|
||||
| Floating point | `float16`, `float32`, `float64` |
|
||||
| String | `utf8`, `large_utf8` |
|
||||
| Binary | `binary`, `large_binary`, `fixed_size_binary(n)` |
|
||||
| Date and time | `date32`, `date64`, `time32`, `time64`, `timestamp` (with or without timezone), `duration` |
|
||||
| Nested | `struct`, `list`, `large_list`, `fixed_size_list`, `map` |
|
||||
| Dictionary | `dictionary(key, value)` |
|
||||
|
||||
Two limits are worth stating because they surface as schema errors rather than
|
||||
as documentation:
|
||||
|
||||
- A `map` with `keys_sorted = true` is rejected. Sorted-key maps do not survive
|
||||
the round trip through the Python bindings, so they are refused rather than
|
||||
silently unsorted.
|
||||
- Not every type is legal as a `fixed_size_list` child. The vector case is,
|
||||
which is what matters in practice.
|
||||
|
||||
## SQL names for Arrow types {#names}
|
||||
|
||||
| SQL | Arrow |
|
||||
@@ -79,6 +102,26 @@ The default reads blob columns as descriptions and other binary columns as
|
||||
binary. A query selecting a blob column and expecting bytes gets a struct.
|
||||
</Warning>
|
||||
|
||||
### bfloat16 {#bfloat16}
|
||||
|
||||
Sixteen-bit floats in the bfloat16 layout, stored as `fixed_size_binary(2)`.
|
||||
Useful as a vector element type when halving the memory matters more than the
|
||||
precision does.
|
||||
|
||||
## Geospatial {#geospatial}
|
||||
|
||||
<Warning>
|
||||
Lance defines geospatial types and `st_*` functions on
|
||||
[GeoArrow](https://geoarrow.org), but they sit behind an optional build feature
|
||||
that **LanceDB does not ship**. Calling `st_intersects` and friends returns an
|
||||
error naming the missing feature rather than an unknown-function error, and the
|
||||
spatial index is gated the same way.
|
||||
</Warning>
|
||||
|
||||
Geometry can still be stored -- it is binary, and binary columns work -- but
|
||||
nothing interprets it. Filter on a bounding box held in ordinary numeric
|
||||
columns if you need spatial predicates today.
|
||||
|
||||
## Pseudo-columns {#pseudo-columns}
|
||||
|
||||
Row metadata, available only when named — so it stays out of `SELECT *`:
|
||||
|
||||
@@ -11,7 +11,7 @@ keywords: ["insert", "update", "delete", "copy", "dml"]
|
||||
Statements that change rows. For the shape of the table, see [DDL](/sql/ddl).
|
||||
|
||||
Each of these commits a new table version, so
|
||||
[versioning](/tables/versioning) and the
|
||||
[versioning](/management/table#versioning) and the
|
||||
[write guarantees](/basics/guarantees#writes) apply -- including that the
|
||||
version advances even when a statement affects zero rows.
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ explain how to read them:
|
||||
<Card title="Explain" icon="map" href="/search/explain">
|
||||
Reading a plan: operators, pushdown, index usage.
|
||||
</Card>
|
||||
<Card title="Explain Analyze" icon="gauge" href="/search/explain-analyze">
|
||||
<Card title="Explain Analyze" icon="gauge" href="/search/explain">
|
||||
Reading the metrics, and a worked before/after.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
@@ -132,4 +132,4 @@ same operators the SDK methods do:
|
||||
EXPLAIN ANALYZE SELECT * FROM documents WHERE identifier < 1000
|
||||
```
|
||||
|
||||
See [Explain](/search/explain) and [Explain Analyze](/search/explain-analyze).
|
||||
See [Explain](/search/explain) and [Explain Analyze](/search/explain).
|
||||
|
||||
@@ -96,5 +96,5 @@ read.
|
||||
<Note>
|
||||
Window functions sort within each partition, so on a large table they are among
|
||||
the more expensive things you can ask for. Filter first, and check the cost with
|
||||
[Explain Analyze](/search/explain-analyze).
|
||||
[Explain Analyze](/search/explain).
|
||||
</Note>
|
||||
|
||||
Reference in New Issue
Block a user