lancedb/docs/web/ingestion/create-table.mdx

---
title: "Creating a table"
sidebarTitle: "Create Table"
description: "Start here: turn the data you already have into a LanceDB table."
icon: table
keywords: ["create table", "ingest", "arrow", "pandas", "polars", "pydantic", "schema"]
availability:
  oss: available
  enterprise: available
  summary: >-
    Tables are created, opened and dropped in the embedded library and on a
    deployment. On a deployment, reusing an existing table does not check its
    schema against your data.
---

import {
    TsConnect,
    RsConnect,
} from '/snippets/connection.mdx';
import {
    PyCreateTableFromDicts as CreateTableFromDicts,
    TsCreateTableFromDicts as TsCreateTableFromDicts,
    RsCreateTableFromDicts as RsCreateTableFromDicts,
    PyCreateTableConflictHandling as CreateTableConflictHandling,
    TsCreateTableConflictHandling as TsCreateTableConflictHandling,
    RsCreateTableConflictHandling as RsCreateTableConflictHandling,
    PyCreateTableFromPandas as CreateTableFromPandas,
    PyCreateTableCustomSchema as CreateTableCustomSchema,
    TsCreateTableCustomSchema as TsCreateTableCustomSchema,
    RsCreateTableCustomSchema as RsCreateTableCustomSchema,
    PyCreateTableFromPolars as CreateTableFromPolars,
    PyCreateTableFromArrow as CreateTableFromArrow,
    TsCreateTableFromArrow as TsCreateTableFromArrow,
    RsCreateTableFromArrow as RsCreateTableFromArrow,
    PyCreateTableFromPydantic as CreateTableFromPydantic,
    PyCreateTableNestedSchema as CreateTableNestedSchema,
    PyOpenExistingTable as OpenExistingTable,
    TsOpenExistingTable as TsOpenExistingTable,
    RsOpenExistingTable as RsOpenExistingTable,
    PyCreateEmptyTable as CreateEmptyTable,
    TsCreateEmptyTable as TsCreateEmptyTable,
    RsCreateEmptyTable as RsCreateEmptyTable,
    PyCreateEmptyTablePydantic as CreateEmptyTablePydantic,
    PyDropTable as DropTable,
    TsDropTable as TsDropTable,
    RsDropTable as RsDropTable,
    PyTablesBasicConnect as TablesBasicConnect,
    PyTablesDocumentModel as TablesDocumentModel,
    PyTablesTzValidator as TablesTzValidator,
} from '/snippets/tables.mdx';

A table is created from data you already have -- a list of objects, an Arrow
table, a pandas or polars frame, Pydantic models -- or from a schema alone when
the rows come later.

## Create a table with data {#create-a-table-with-data}

Initialize a LanceDB connection and create a table

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {TablesBasicConnect}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsConnect}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsConnect}
    </CodeBlock>
</CodeGroup>

Depending on the SDK, LanceDB can ingest arrays of records, Arrow tables or record batches, and Arrow batch iterators or readers. Let's take a look at some of the common patterns.

### From list of objects {#from-list-of-objects}

You can provide a list of objects to create a table. The Python and TypeScript SDKs
support lists/arrays of dictionaries, while the Rust SDK supports lists of structs.
In Python, pass a list or other batch-like object; a single bare `dict` or single
`LanceModel` is rejected.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateTableFromDicts}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsCreateTableFromDicts}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsCreateTableFromDicts}
    </CodeBlock>
</CodeGroup>

### Handle existing tables {#handle-existing-tables}

By default, `create_table` raises an error if a table with the same name already exists.
You can change this behavior with two parameters that resolve the conflict in different ways:

- **Idempotent open**: return the existing table without modifying it. Use when your
  code may run more than once (notebooks, retries, init scripts) and you want to reuse
  the table on subsequent runs. The provided data is ignored. In the embedded
  library the schema is validated against the existing table and a mismatch
  raises an error; a deployment returns the existing table without comparing
  schemas.
- **Overwrite**: drop the existing table and create a new one with the provided data.
  Use this for test fixtures or when you intentionally want to replace prior contents.
  This permanently discards the old table's data.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateTableConflictHandling}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsCreateTableConflictHandling}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsCreateTableConflictHandling}
    </CodeBlock>
</CodeGroup>

<Note>
`exist_ok` / `existOk` does not append the provided data to an existing table. Use
[`table.add()`](/ingestion/insert) for that. If you need to ensure a table exists *and*
contains specific rows, prefer the [empty-table-then-add pattern](#create-empty-table).
</Note>

### From a custom schema {#from-a-custom-schema}

You can define a custom Arrow schema for the table. This is useful when you want to have more control over the column types and metadata.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateTableCustomSchema}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsCreateTableCustomSchema}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsCreateTableCustomSchema}
    </CodeBlock>
</CodeGroup>

An explicit schema is also where you control nullability. If later writes omit a
non-nullable column, or provide actual nulls for it, ingestion fails; nullable columns can be
omitted or written with null values. Without an explicit schema, Python infers list-like vector
values as fixed-size `float32` vector fields from the observed dimension.

For Python ingest, malformed vector values fail by default. If you expect occasional wrong-length,
null, or NaN vectors, choose an `on_bad_vectors` policy: `"drop"` removes those rows, `"fill"` writes
`fill_value`, and `"null"` writes nulls.

### From an Arrow Table {#from-an-arrow-table}
You can also create LanceDB tables directly from Arrow tables.
Rust uses an Arrow `RecordBatchReader` for the same Arrow-native ingest flow.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateTableFromArrow}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsCreateTableFromArrow}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsCreateTableFromArrow}
    </CodeBlock>
</CodeGroup>


### From a Pandas DataFrame {#from-a-pandas-dataframe}
<Badge color="green">Python Only</Badge>

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateTableFromPandas}
    </CodeBlock>
</CodeGroup>

<Note title="Note">
Data is converted to Arrow before being written to disk. For maximum control over how data is saved, either provide the PyArrow schema to convert to or else provide a PyArrow Table directly.
</Note>

<Note title="Vector Column Type">
The **`vector`** column needs to be a [Vector](/integrations/data/pydantic#vector-field) (defined as [pyarrow.FixedSizeList](https://arrow.apache.org/docs/python/generated/pyarrow.list_.html)) type.
</Note>

### From a Polars DataFrame {#from-a-polars-dataframe}
<Badge color="green">Python Only</Badge>

LanceDB supports [Polars](https://pola.rs/), a modern, fast DataFrame library
written in Rust. Just like in Pandas, the Polars integration is enabled by PyArrow
under the hood. A deeper integration between LanceDB Tables and Polars DataFrames
is on the way.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateTableFromPolars}
    </CodeBlock>
</CodeGroup>

### From Pydantic Models {#from-pydantic-models}
<Badge color="green">Python Only</Badge>

When you create an empty table without data, you must specify the table schema.
LanceDB supports creating tables by specifying a PyArrow schema or a specialized
Pydantic model called `LanceModel`.

For example, the following Content model specifies a table with 5 columns:
`movie_id`, `vector`, `genres`, `title`, and `imdb_id`. When you create a table, you can
pass the class as the value of the `schema` parameter to `create_table`.
The `vector` column is a `Vector` type, which is a specialized Pydantic type that
can be configured with the vector dimensions. It is also important to note that
LanceDB only understands subclasses of `lancedb.pydantic.LanceModel`
(which itself derives from `pydantic.BaseModel`).

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateTableFromPydantic}
    </CodeBlock>
</CodeGroup>

#### Nested schemas {#nested-schemas}

Sometimes your data model may contain nested objects. For example, you may want to store the document string and the document source name as a nested Document object:

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {TablesDocumentModel}
    </CodeBlock>
</CodeGroup>

This can be used as the type of a LanceDB table column:

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateTableNestedSchema}
    </CodeBlock>
</CodeGroup>

This creates a struct column called "document" that has two subfields
called "content" and "source":

```bash
In [28]: tbl.schema
Out[28]:
id: string not null
vector: fixed_size_list<item: float>[1536] not null
    child 0, item: float
document: struct<content: string not null, source: string not null> not null
    child 0, content: string not null
    child 1, source: string not null
```

#### Validators {#validators}

Because `LanceModel` inherits from Pydantic's `BaseModel`, you can combine them with Pydantic's
[field validators](https://docs.pydantic.dev/latest/concepts/validators). The example
below shows how to add a validator to ensure that only valid timezone-aware datetime objects are used
for a `created_at` field.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {TablesTzValidator}
    </CodeBlock>
</CodeGroup>

When you run this code it, should raise the `ValidationError`.

<Note>
Datasets too large to pass to `create_table` at once are covered separately --
see [Insert data](/ingestion/insert#large-datasets), which explains why
`table.add()` parallelizes where `create_table` does not.
</Note>

## Create empty table {#create-empty-table}
You can create an empty table for scenarios where you want to add data to the table later.
An example would be when you want to collect data from a stream/external file and then add it to a table in
batches.

An empty table can be initialized via an Arrow schema.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateEmptyTable}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsCreateEmptyTable}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsCreateEmptyTable}
    </CodeBlock>
</CodeGroup>

Alternatively, you can also use Pydantic to specify the schema for the empty table. Note that we do not
directly import `pydantic` but instead use `lancedb.pydantic` which is a subclass of `pydantic.BaseModel`
that has been extended to support LanceDB specific types like `Vector`.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateEmptyTablePydantic}
    </CodeBlock>
</CodeGroup>

Once the empty table has been created, you can append to it or modify its contents,
as explained in [Ingesting data](/ingestion).

## Open an existing table {#open-an-existing-table}

You can open an existing table by specifying the name of the table to the `open_table` / `openTable` method.
If you forget the name of your table, you can always get a listing of all table names.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {OpenExistingTable}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsOpenExistingTable}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsOpenExistingTable}
    </CodeBlock>
</CodeGroup>

## Drop a table {#drop-a-table}

Use the `drop_table()` method on the database to remove a table.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {DropTable}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsDropTable}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsDropTable}
    </CodeBlock>
</CodeGroup>

This permanently removes the table and is not recoverable, unlike deleting rows.
By default, if the table does not exist an exception is raised. To suppress this
in Python on a local database, you can pass in `ignore_missing=True`.