--- title: "Creating a table" sidebarTitle: "Create Table" description: "Create a table from records or a schema and choose input formats." icon: table keywords: ["create table", "ingest", "arrow", "pandas", "polars", "pydantic", "schema"] availability: oss: available enterprise: available summary: >- Create and open tables in OSS and Enterprise. --- import { TsConnect, RsConnect, } from '/snippets/connection.mdx'; import { TsCreateTableFromDicts as TsCreateTableFromDicts, RsCreateTableFromDicts as RsCreateTableFromDicts, TsCreateTableConflictHandling as TsCreateTableConflictHandling, RsCreateTableConflictHandling as RsCreateTableConflictHandling, TsCreateTableCustomSchema as TsCreateTableCustomSchema, RsCreateTableCustomSchema as RsCreateTableCustomSchema, TsCreateTableFromArrow as TsCreateTableFromArrow, RsCreateTableFromArrow as RsCreateTableFromArrow, TsOpenExistingTable as TsOpenExistingTable, RsOpenExistingTable as RsOpenExistingTable, TsCreateEmptyTable as TsCreateEmptyTable, RsCreateEmptyTable as RsCreateEmptyTable, TsDropTable as TsDropTable, RsDropTable as RsDropTable, PyTablesBasicConnect as TablesBasicConnect, PyTablesDocumentModel as TablesDocumentModel, PyTablesTzValidator as TablesTzValidator, } from '/snippets/tables.mdx'; import { PyCreateTableFromDicts as CreateTableFromDicts, PyCreateTableConflictHandling as CreateTableConflictHandling, PyCreateTableFromPandas as CreateTableFromPandas, PyCreateTableCustomSchema as CreateTableCustomSchema, PyCreateTableFromPolars as CreateTableFromPolars, PyCreateTableFromArrow as CreateTableFromArrow, PyCreateTableFromPydantic as CreateTableFromPydantic, PyCreateTableNestedSchema as CreateTableNestedSchema, PyOpenExistingTable as OpenExistingTable, PyCreateEmptyTable as CreateEmptyTable, PyCreateEmptyTablePydantic as CreateEmptyTablePydantic, PyDropTable as DropTable, } from '/snippets/ingestion_examples.mdx'; import { PyWriteWorkflow, TsWriteWorkflowBody, TsWriteWorkflowTail, RsWriteWorkflow } from '/snippets/ingestion_examples.mdx'; Create a table from existing records, or define an empty table before data arrives. Define a schema to specify column types, allow or disallow missing values, and set the size of your vectors. ## Create a table with data {#create-a-table-with-data} The Python input examples below are complete programs. Install `lancedb` and any additional libraries imported by the example, save one as a Python file, and run it. Each uses its own temporary local database. The TypeScript and Rust blocks illustrate the corresponding input formats. For a complete connection-to-result program in either language, expand [the write workflow](#complete-example). Table names follow the [naming rules](/basics/naming). ### From list of objects {#from-list-of-objects} You can provide a list of objects to create a table. The Python and TypeScript SDKs support lists/arrays of records. The Rust example converts structs to Arrow batches. In Python, pass a list or other batch-like object; a single bare `dict` or single `LanceModel` is rejected. {CreateTableFromDicts} {TsCreateTableFromDicts} {RsCreateTableFromDicts} ### If the table already exists {#handle-existing-tables} By default, creating a table fails if its name is already in use. Choose whether to open that table or overwrite it. #### Open the existing table Use `exist_ok=True` in Python or `existOk: true` in TypeScript to return the existing table without changing its contents. This is useful for setup code that may run more than once. Any data supplied to the create call is ignored when the table already exists. Inspect the returned table's schema if your code depends on specific columns or types. Opening an existing table does not change its schema to match the data supplied in the create call. #### Overwrite the table Use `mode="overwrite"` in Python or `mode: "overwrite"` in TypeScript to replace the existing table with the supplied data and schema. Choose this only when you intend to replace its previous contents. The following Python program demonstrates both choices: {CreateTableConflictHandling} `exist_ok` / `existOk` does not append the provided data to an existing table. Use [`table.add()`](/ingestion/insert) for that. If you need to ensure a table exists *and* contains specific rows, choose an explicit [merge policy](/ingestion/merge-insert). Creating an empty table and repeatedly appending the same batch does not deduplicate it. ### From a custom schema {#from-a-custom-schema} You can define a custom Arrow schema for the table. This is useful when you want to have more control over the column types and metadata. {CreateTableCustomSchema} {TsCreateTableCustomSchema} {RsCreateTableCustomSchema} An explicit schema also determines which columns accept missing values (`null`). If later writes omit a non-nullable column, or provide actual nulls for it, ingestion fails; nullable columns can be omitted or written with null values. Without an explicit schema, Python infers list-like vector values as fixed-size `float32` vector fields from the observed dimension. For Python ingest, malformed vector values fail by default. If you expect occasional wrong-length, null, or NaN vectors, choose an `on_bad_vectors` policy: `"drop"` removes those rows, `"fill"` writes `fill_value`, and `"null"` writes nulls. ### From an Arrow Table {#from-an-arrow-table} You can also create LanceDB tables directly from Arrow tables. Rust uses an Arrow `RecordBatchReader` for the same Arrow-native ingest flow. {CreateTableFromArrow} {TsCreateTableFromArrow} {RsCreateTableFromArrow} ### From a Pandas DataFrame {#from-a-pandas-dataframe} Python Only {CreateTableFromPandas} Data is converted to Arrow before being written to disk. For maximum control over how data is saved, either provide the PyArrow schema to convert to or else provide a PyArrow Table directly. The **`vector`** column needs to be a [Vector](/integrations/data/pydantic#vector-field) (defined as [pyarrow.FixedSizeList](https://arrow.apache.org/docs/python/generated/pyarrow.list_.html)) type. ### From a Polars DataFrame {#from-a-polars-dataframe} Python Only Pass a Polars DataFrame as input. Install `polars` for this example; the conversion preserves the table structure through Arrow. {CreateTableFromPolars} ### From Pydantic Models {#from-pydantic-models} Python Only When you create an empty table without data, you must specify the table schema. LanceDB supports creating tables by specifying a PyArrow schema or a specialized Pydantic model called `LanceModel`. For example, the following Content model specifies a table with 5 columns: `movie_id`, `vector`, `genres`, `title`, and `imdb_id`. When you create a table, you can pass the class as the value of the `schema` parameter to `create_table`. The `vector` column is a `Vector` type, which is a specialized Pydantic type that can be configured with the vector dimensions. It is also important to note that LanceDB only understands subclasses of `lancedb.pydantic.LanceModel` (which itself derives from `pydantic.BaseModel`). {CreateTableFromPydantic} #### Nested schemas {#nested-schemas} Sometimes your data model may contain nested objects. For example, you may want to store the document string and the document source name as a nested Document object: {TablesDocumentModel} This can be used as the type of a LanceDB table column: {CreateTableNestedSchema} This creates a struct column called "document" that has two subfields called "content" and "source": ```bash In [28]: tbl.schema Out[28]: id: string not null vector: fixed_size_list[1536] not null child 0, item: float document: struct not null child 0, content: string not null child 1, source: string not null ``` #### Validators {#validators} Because `LanceModel` inherits from Pydantic's `BaseModel`, you can combine them with Pydantic's [field validators](https://docs.pydantic.dev/latest/concepts/validators). The example below shows how to add a validator to ensure that only valid timezone-aware datetime objects are used for a `created_at` field. {TablesTzValidator} The example catches a `ValidationError` for the timestamp with a different timezone. A validator runs when the Pydantic model is constructed; declaring a model as the table schema does not make every later write run that validator. Datasets too large to pass to `create_table` at once are covered separately; see [Insert data](/ingestion/insert#large-datasets) for batches, file-backed input and write parallelism. ## Create empty table {#create-empty-table} You can create an empty table for scenarios where you want to add data to the table later. An example would be when you want to collect data from a stream/external file and then add it to a table in batches. An empty table can be initialized via an Arrow schema. {CreateEmptyTable} {TsCreateEmptyTable} {RsCreateEmptyTable} Alternatively, define the schema with `LanceModel`, a Pydantic model class provided by `lancedb.pydantic`. Its `Vector` type declares a fixed vector dimension. {CreateEmptyTablePydantic} Once the empty table has been created, you can append to it or modify its contents, as explained in [Ingesting data](/ingestion). ## Open an existing table {#open-an-existing-table} You can open an existing table by specifying the name of the table to the `open_table` / `openTable` method. If you forget the name of your table, you can always get a listing of all table names. {OpenExistingTable} {TsOpenExistingTable} {RsOpenExistingTable} ## Drop a table {#drop-a-table} Use the `drop_table()` method on the database to remove a table. {DropTable} {TsDropTable} {RsDropTable} Dropping a table removes the table object. Deleting rows keeps the table and changes its contents; retained versions and cleanup are described in [Cleanup](/maintenance/cleanup). On a local database, dropping a table that does not exist raises an error; in Python, pass `ignore_missing=True` to suppress it. A deployment may report success instead, depending on how it stores its catalog, so do not rely on the error to tell you a table was missing. Python's deployment connection does not take `ignore_missing`. ## Complete write workflow {#complete-example} Each program creates a temporary local database, performs the five operations, prints the result, and removes its temporary data when finished. - **Python:** install `lancedb`, save the code as `ingestion.py`, and run `python ingestion.py`. - **TypeScript:** install `@lancedb/lancedb`, save the code as `ingestion.mts`, and run `node ingestion.mts` with Node.js 22.18 or later. - **Rust:** download the [complete Rust project](/static/code/ingestion-rust.zip), extract it, and run `cargo run --locked`. It includes `src/main.rs`, the Cargo manifest and a lockfile for the example's dependencies. The code is shown below. {PyWriteWorkflow} {TsWriteWorkflowBody} {TsWriteWorkflowTail} {RsWriteWorkflow} After the operations, Alice has 10 logins, Bobby has 21, and Charlie has 6. Dana is selected for deletion and is absent from the final result.