c269524b2f
This opens up the door for more custom database implementations than the
two we have today. The biggest change should be inivisble:
`ConnectionInternal` has been renamed to `Database`, made public, and
refactored
However, there are a few breaking changes. `data_storage_version` and
`enable_v2_manifest_paths` have been moved from options on
`create_table` to options for the database which are now set via
`storage_options`.
Before:
```
db = connect(uri)
tbl = db.create_table("my_table", data, data_storage_version="legacy", enable_v2_manifest_paths=True)
```
After:
```
db = connect(uri, storage_options={
"new_table_enable_v2_manifest_paths": "true",
"new_table_data_storage_version": "legacy"
})
tbl = db.create_table("my_table", data)
```
BREAKING CHANGE: the data_storage_version, enable_v2_manifest_paths
options have moved from options to create_table to storage_options.
BREAKING CHANGE: the use_legacy_format option has been removed,
data_storage_version has replaced it for some time now
1.8 KiB
@lancedb/lancedb • Docs
@lancedb/lancedb / CreateTableOptions
Interface: CreateTableOptions
Properties
dataStorageVersion?
optional dataStorageVersion: string;
The version of the data storage format to use.
The default is stable.
Set to "legacy" to use the old format.
Deprecated
Pass new_table_data_storage_version to storageOptions instead.
embeddingFunction?
optional embeddingFunction: EmbeddingFunctionConfig;
enableV2ManifestPaths?
optional enableV2ManifestPaths: boolean;
Use the new V2 manifest paths. These paths provide more efficient opening of datasets with many versions on object stores. WARNING: turning this on will make the dataset unreadable for older versions of LanceDB (prior to 0.10.0). To migrate an existing dataset, instead use the LocalTable#migrateManifestPathsV2 method.
Deprecated
Pass new_table_enable_v2_manifest_paths to storageOptions instead.
existOk
existOk: boolean;
If this is true and the table already exists and the mode is "create" then no error will be raised.
mode
mode: "overwrite" | "create";
The mode to use when creating the table.
If this is set to "create" and the table already exists then either an error will be thrown or, if existOk is true, then nothing will happen. Any provided data will be ignored.
If this is set to "overwrite" then any existing table will be replaced.
schema?
optional schema: SchemaLike;
storageOptions?
optional storageOptions: Record<string, string>;
Configuration for object storage.
Options already set on the connection will be inherited by the table, but can be overridden here.
The available options are described at https://lancedb.github.io/lancedb/guides/storage/